CODEWITHSUNDEEP
cassemblyx86returncalling-convention
Medals
Medals

Reputation: 141

how C compiler treats a struct return value from a function, in ASM

When speaking about C function's return value, the return value is stored in the EAX register. Suppose we are speaking about 32 bit register, integers are welcomed, but what happens when we return these types: long long, long double, a struct/union that is larger than 32bit.

Upvotes: 13

Views: 4739

Answers (2)

Peter Cordes
Peter Cordes

Reputation: 366095

In common x86 calling conventions, objects that fit in two registers are returned in RDX:RAX. This is the same register pair that is an implicit input/output for div and mul instructions, and for cdq / cqo (sign extend e/rax into e/rdx).

The i386 Linux (SysV) calling convention only returns 64bit integers that way. Structs (even a struct consisting of a single int32_t) use the hidden-parameter method instead of being packed eax or edx:eax. 64bit Linux, and Microsoft's current standard __vectorcall, both pack structs into e/rax, or e/rdx:e/rax.

Many calling conventions handle larger objects by adding a hidden extra parameter: a pointer to space for storing the return value. Consult the ABI documentation for the specific ABI you are using. (links at in the wiki).

Compared to other calling conventions discussed in comments (e.g. implicitly using space on the stack to store large objects being returned), passing a pointer can save a copy, because the pointer can point to the final destination instead of scratch space on the stack.

But usually only if the final destination is on the stack. The callee can assume the return-value object is not the same object as / doesn't alias any global or any memory it can reach via pointers. i.e. the caller has to do escape analysis. See What prevents the usage of a function argument as hidden pointer? - In the C abstract machine, the return value object isn't written until the moment the function is returning, after it's done reading anything.

But it can point to whatever convenient place in the caller's stack frame, not a fixed place relative to RSP, which may avoid a copy.

Upvotes: 11

user1129665
user1129665

Reputation:

Consider this program:

struct object_t
{
  int m1;
  int m2;
  int m3;
};

struct object_t
test1(void)
{
  struct object_t o = {1, 2, 3};
  return o;
}

long long
test2(void)
{
  return 0LL;
}

long double
test3(void)
{
  return 0.0L;
}

compiled on Windows with (object file, minimum instructions, with no x87 instructions):

$ gcc -Wall -c -O2 -mno-80387 test.c -o test.o

The first function:

00000000 <_test1>:
   0:   8b 44 24 04             mov    eax,DWORD PTR [esp+0x4]
   4:   c7 00 01 00 00 00       mov    DWORD PTR [eax],0x1
   a:   c7 40 04 02 00 00 00    mov    DWORD PTR [eax+0x4],0x2
  11:   c7 40 08 03 00 00 00    mov    DWORD PTR [eax+0x8],0x3
  18:   c3                      ret

The caller will provide a pointer to where his structure is over the stack as first argument and test1 will fill it using that pointer.

Second function (sizeof(long long) == 8):

00000020 <_test2>:
  20:   31 c0                   xor    eax,eax
  22:   31 d2                   xor    edx,edx
  24:   c3                      ret

The result will be returned over two registers eax and edx, not just eax.

Third function (sizeof(long double) == 12):

00000030 <_test3>:
  30:   31 c0                   xor    eax,eax
  32:   31 d2                   xor    edx,edx
  34:   31 c9                   xor    ecx,ecx
  36:   c3                      ret

The return value will be passed over three registers, eax, edx, ecx.

Upvotes: 6

Related Questions