CODEWITHSUNDEEP
windowsassemblywinapix86-64nasm
Erik Zimmerman
Erik Zimmerman

Reputation: 29

How To Properly call 64 Bit Windows API In Assembly

Using NASM and Mingw-w64 I've been trying to run the following program which is supposed to print a message to the screen using the Windows API. It runs, but nothing shows on the console and it results in an invalid access to memory location (error code 0x3e6h). Why is that, and how can I get the program to run properly?

global main
extern ExitProcess
extern GetStdHandle
extern WriteFile

section .text
main:
    mov     rcx,                  0fffffff5h
    call    GetStdHandle

    mov     rcx,                  rax
    mov     rdx,                  NtlpBuffer
    mov     r8,                   NtnNBytesToWrite
    mov     r9,                   NtlpNBytesWritten
    mov     dword [rsp - 04h],    00h
    call    WriteFile
ExitProgram:
    mov     rcx,                  00h
    call    ExitProcess

section .data
NtlpBuffer:        db    'Hello, World!', 00h
NtnNBytesToWrite:  dd    0eh

section .bss
NtlpNBytesWritten: resd  01h

Compiled by

nasm -f win64 test.asm
gcc -s -o test.exe test.obj

Upvotes: 2

Views: 4608

Answers (1)

Jester
Jester

Reputation: 58822

[rsp-04h] is addressing below the stack pointer, that's a bad idea. Whatever you write there will be overwritten by the call anyway. Looks like you need to brush up on your knowledge of the calling convention. Shadow space for the 4 arguments in registers have to be allocated and further arguments must be placed on top of them.

Also, the number of bytes to write should be the actual number, not a pointer.

global main
extern GetStdHandle
extern WriteFile

section .text
main:
    sub     rsp, 40          ; reserve shadow space and align the stack by 16
    mov     ecx, -11         ; GetStdHandle takes a DWORD arg, write it as 32-bit.  This is STD_OUTPUT_HANDLE
    call    GetStdHandle

    mov     rcx, rax
    mov     rdx, NtlpBuffer         ; or better, lea rdx, [rel NtlpBuffer]
    mov     r8, [NtnNBytesToWrite]  ; or better, make it an EQU constant for mov r8d, bytes_to_write
    mov     r9, NtlpNBytesWritten   ; first 4 args in regs
    mov     qword [rsp + 32], 00h   ; fifth arg on the stack above the shadow space.  Also, this is a pointer so it needs to be a qword store.
    call    WriteFile
    add     rsp, 40
ExitProgram:
    xor     eax, eax
    ret

section .data
NtlpBuffer:        db    'Hello, World!', 00h
NtnNBytesToWrite:  dq    0eh

section .bss
NtlpNBytesWritten: resd  01h

See How to load address of function or label into register re: using RIP-relative LEA instead of mov reg, imm64 to put addresses into registers.

The trailing 0 byte in the string isn't needed; WriteFile takes pointer + length, not an implicit-length C string. You might want to use db 'Hello...', 13,10 to include a newline. (DOS CR LF.)

On function entry, RSP%16 == 8. sub rsp, 40 will reserve the 32 bytes of shadow space, and an extra 8 bytes to make RSP%16 == 0 since we don't push anything. This gets us ready for another call that pushes an 8-byte return address which will make RSP%16 == 8 on entry to the callee.

See also Hello World using NASM (and GoLink) on x64 64 bit Windows for a tidied up version of this Hello World using RIP-relative LEA, less static storage, and not hard-coding the string length.

Upvotes: 6

Related Questions