Calling printf() in assembly causes a 'floating point exception'

Question

I have the following code:

extern printf

section .data

A db 1
B db 2
C db 3
D db 4
E db 5

fmt  db "%d",10,0
fmt2 db "An overflow has occured",10,0

section .text

    global _start

_start:
    xor rax, rax
    xor rbx, rbx
    xor rdx, rdx

    xor ax, ax
    mov al, [B]
    mul ax
    jo over
    call printax
    mov bx, ax

    xor ax, ax
    mov al, [C]
    mul ax
    jo over
    call printax

    xor cx, cx
    mov cl, [C]
    mul cx
    jo over
    call printax

    xor cx, cx
    mov cl, [E]
    div cx
    jo over
    call printax
    xor dx, dx

    xor al, al
    mov rdi, fmt
    call printf
    jmp exit

over:
    mov rdi, fmt2
    xor al, al
    call printf

exit:
    mov rax, 60
    mov rdi, 0
    syscall

printbx:
    xor rsi, rsi
    mov si, bx
    mov cl, al
    xor al, al
    mov rdi, fmt
    call printf
    mov al, cl
    xor si, si
    ret

printax:
    xor rsi, rsi
    mov si, ax
    xor al, al
    mov rdi, fmt
    call printf
    mov ax, si
    xor si, si
    ret

I compile it with nasm -f elf64 1.asm and link it using ld -dynamic-linker /lib/ld-linux-x86-64.so.2 1.o -o 1 -lc. When I execute the binary, I get

$ ./1
4
9
0
Floating point exception (core dumped)

printf() doesn't always fail when I call it in assembly code.

---

Removing printf() calls from printax and printbx gives me

$ ./1
0

---

Update: The exception also disappears if I remove the div cx line. Then I get the following output:

$./1
4
9
0
0
0

But it doesn't disappear even if I add

mov cx, 1
mov ax, 1

before div cx.

Answer 1

Take another look at the x86-64 ABI calling conventions. Specifically, on page 15:

Registers %rbp, %rbx and %r12 through %r15 “belong” to the calling function and the called function is required to preserve their values. In other words, a called function must preserve these registers’ values for its caller. Remaining registers “belong” to the called function. If a calling function wants to preserve such a register value across a function call, it must save the value in its local stack frame.

So when you call printf, you must assume that all registers except rbp, rbx, r12..r15 are clobbered. This has two effects for your program:

• You attempt to save ax around the call to printf by stashing its value in the si register and then putting it back later, but this doesn't help because si can be clobbered.

• Your div cx instruction divides the contents of dx:ax by cx, but dx might also have been clobbered.

The latter is the specific cause of the SIGFPE (which despite its name is also raised on an integer divide overflow), at least in my test runs. After the printf returns, dx contains some huge number, such that dx:ax divided by cx does not fit in 16 bits, which is an overflow.

(This also explains why the crash went away when you took out the printf call - it was no longer there to clobber your registers.)

The other lesson here is that you can't check for divide overflow on x86 by doing a jo afterwards; this error is primarily signaled by an exception, not by flags. So you really have to check your operands before executing the instruction, or else arrange to handle the exception if it occurs (which is more complicated and beyond the scope of this answer).

A couple of other notes:

• Before your final call to printf (just before jmp exit), you don't load anything into rsi, so the value printed is garbage.

• 16-bit arithmetic is usually not to be preferred on x86-32 or x86-64 unless there is a really good reason for it. It is no faster and bloats your code with operand size prefixes. Better to do all your work with 32-bit arithmetic.

• Since you're using your own entry point rather than letting the C library call your main, this means that libc has not had the opportunity to run its own initialization code. Therefore, it is not necessarily safe to call any libc function, particularly stdio and allocation functions. It seems that printf happens to work okay in this case, and maybe it is all right for debugging, but you shouldn't plan to write your program this way for production.