CODEWITHSUNDEEP
winapiassembly64-bitx86-64nasm
christian_schmidt
christian_schmidt

Reputation: 30

Repeated call of WriteConsole (NASM x64 on Win64)

I started to learn Assembly lately and for practice, I thought of makeing a small game. To make the border graphic of the game I need to print a block character n times. To test this, I wrote the following code:

bits 64

global main
extern ExitProcess
extern GetStdHandle
extern WriteConsoleA

section .text
    
main:
    mov rcx, -11
    call GetStdHandle   
    mov rbx, rax
drawFrame:
    mov r12, [sze]
    l:
    mov rcx, rbx
    mov rdx, msg
    mov r8, 1
    sub rsp, 48
    mov r9, [rsp+40]
    mov qword [rsp+32], 0
    call WriteConsoleA
    dec r12
    jnz l
    
    xor rcx, rcx
    call ExitProcess

section .data
    score dd 0
    sze dq 20
    msg db 0xdb

I wanted to make this with the WinAPI Function for ouput. Interestingly, this code stops after printing one char when using WriteConsoleA, but when I use C's putchar, it works correctly. I could also manage to make a C equivalent with the WriteConsoleA function, which also works fine. The disassembly of the C code didn't bring me further.

I suspect there's something wrong in my use of the stack that I don't see. Hopefully someone can explain or point out.

Upvotes: 1

Views: 538

Answers (1)

Michael Petch
Michael Petch

Reputation: 47593

You don't want to keep subtracting 48 from RSP through each loop. You only need to allocate that space once before the loop and before you call a C library function or the WinAPI.

The primary problem is with your 4th parameter in R9. The WriteConsole function is defined as:

BOOL WINAPI WriteConsole(
  _In_             HANDLE  hConsoleOutput,
  _In_       const VOID    *lpBuffer,
  _In_             DWORD   nNumberOfCharsToWrite,
  _Out_opt_        LPDWORD lpNumberOfCharsWritten,
  _Reserved_       LPVOID  lpReserved
);

R9 is supposed to be a pointer to a memory location that returns a DWORD with the number of characters written, but you do:

mov r9, [rsp+40]

This moves the 8 bytes starting at memory address RSP+40 to R9. What you want is the address of [rsp+40] which can be done using the LEA instruction:

lea r9, [rsp+40]

Your code could have looked like:

bits 64

global main
extern ExitProcess
extern GetStdHandle
extern WriteConsoleA

section .text
    
main:
    sub rsp, 56          ; Allocate space for local variable(s)
                         ; Allocate 32 bytes of space for shadow store
                         ; Maintain 16 byte stack alignment for WinAPI/C library calls
                         ; 56+8=64 . 64 is evenly divisible by 16.
    mov rcx, -11
    call GetStdHandle   
    mov rbx, rax
drawFrame:
    mov r12, [sze]
l:
    mov rcx, rbx
    mov rdx, msg
    mov r8, 1
    lea r9, [rsp+40]
    mov qword [rsp+32], 0
    call WriteConsoleA
    dec r12
    jnz l
    
    xor rcx, rcx
    call ExitProcess

section .data
    score dd 0
    sze dq 20
    msg db 0xdb

Important Note: In order to be compliant with the 64-bit Microsoft ABI you must maintain the 16 byte alignment of the stack pointer prior to calling a WinAPI or C library function. Upon calling the main function the stack pointer (RSP) was 16 byte aligned. At the point the main function starts executing the stack is misaligned by 8 because the 8 byte return address was pushed on the stack. 48+8=56 doesn't get you back on a 16 byte aligned stack address (56 is not evenly divisible by 16) but 56+8=64 does. 64 is evenly divisible by 16.

Upvotes: 3

Related Questions