CODEWITHSUNDEEP
gccassemblyx86-64calling-conventionabi
dawidmyk
dawidmyk

Reputation: 71

Changing calling convention in gcc/g++ abi

How could I enforce gcc/g++ to not use registers but only stack in x86_64 to pass arguments to functions, like it was in 32-bit version (and possibly take the function result this way). I know it breaks official ABI and both the caller side and the called side must be compiled this way so it works. I don't care if push/pop or mov/sub way is used. I expect there should be a flag to compiler that could enforce it but I couldn't find it.

Upvotes: 1

Views: 2476

Answers (1)

Peter Cordes
Peter Cordes

Reputation: 365277

It seems you can't without hacking GCC's source code.

There is no standard x86-64 calling convention that uses inefficient stack args.

GCC only knows how to use standard calling conventions, in this case x86-64 SysV and MS Windows fastcall and vectorcall. (e.g. __attribute__((ms_abi)) or vectorcall). Normally nobody wants this; MS's calling convention is already friendly enough for wrappers or variadic functions. You can use that for some functions (controlled by __attribute__) even when compiling for Linux, MacOS, *BSD, etc., if that helps. Hard to imagine a use-case for pure stack args.

GCC lets you specify a register as fixed (never touched by GCC, like -ffixed-rdi), call-clobbered, or call-preserved. But using those with arg-passing registers just creates wrong code, not what you want.

e.g.

int foo(int a, int b, int c);

void caller(int x) {
    foo(1,2,3);
    //foo(4,x,6);
}

compiled by gcc9.2 -O3 -fcall-saved-rdi

caller:
        push    rdi
        mov     edx, 3
        mov     esi, 2
        pop     rdi
        jmp     foo

It saves/restores RDI but doesn't put a 1 in it before calling foo.

And it doesn't leave RDI out of the arg-passing sequence and bump other args later. (I was thinking you might be able to invent a calling convention where all the arg-passing registers were fixed or call-saved, maybe getting GCC to fall back to stack args. But nope.)

Upvotes: 3

Related Questions