CODEWITHSUNDEEP
argumentsparameter-passingx86-64system-calls
Captain Trojan
Captain Trojan

Reputation: 2921

Example of an x86_64 system call which reads parameters from the stack or from fixed memory locations

Studying OS basics (in the context of x86_64), I've learned that system call arguments are most commonly passed through registers, such as RDI, RSI, RDX etc., however, my source claims that it is also possible to pass arguments to the kernel through 1. the stack and 2. using fixed memory locations. Can you give an example of such system calls? I've managed to found none.

Upvotes: 0

Views: 489

Answers (1)

Peter Cordes
Peter Cordes

Reputation: 365277

I don't know about Windows; maybe it does something different.

Linux only ever uses 6 registers for system-call args, not fixed or user-stack locations. If a system call needs more things, one of the args will be a pointer to a struct (like clone3). I think most other x86-64 OSes that use the x86-64 System V ABI are similar. (i.e. all non-Windows one.)

Linux with sysenter from 32-bit user-space may look at the user-space stack for something, but I think just what it needs to be able to return to user-space, not args per-se.

*BSD and MacOS with 32-bit int 0x80 read args from user stack memory, instead of registers, but for 64-bit code they use the x86-64 System V ABI the way Linux does.

The *BSD int 0x80 convention of reading from user ESP is optimized for libc system-call wrappers: it looks for the first arg at 4(%esp), leaving room for a return address at 0(%esp). So the libc wrapper for most system calls could just be int $0x80 / ret, because i386 System V uses a stack-args calling convention.

Obviously it's possible to make a system-calling convention that isn't exclusively register-based, like *BSD in 32-bit mode. It means extra checking, though, since the kernel can't trust any pointers from user-space, not even RSP. For example, mov rsp, 0xffffff...1230 / syscall could try to trick the kernel into reading args from somewhere in kernel space, with the error return value maybe telling you something about what they were. Or causing an invalid page fault if you pass a bad address (or GPF for a non-canonical address).

So it's less convenient. But of course a kernel needs to be able to sanity-check pointer args to syscalls because many like read do take pointers to user-space memory. Still, having to do that on every system call, even ones that should be simpler, is less good.

Register args also lets hand-written asm set up args for a C function safely without needing to do any address checking. Or in modern Linux, just pass a pointer to the register-save area, with C code deciding how many and what width to load. I guess this makes Spectre and ROP attacks harder by not letting user-space enter the kernel with so many user-controlled values in registers for system calls that don't take 6x 64-bit args.

OTOH, with args all on the user stack, an asm entry point just has to pass the user stack pointer to some C function that does the checking and loading.

Upvotes: 1

Related Questions