NASM ReadConsoleA or WriteConsoleA Buffer Debugging Issue

Question

I am writing a NASM Assembly program on Windows to get the user to enter in two single digit numbers, add these together and then output the result. I am trying to use the Windows API for input and output.

Unfortunately, whilst I can get it to read in one number as soon as the program loops round to get the second the program ends rather than asking for the second value.

The output of the program shown below:

enter image description here

What is interesting is that if I input 1 then the value displayed is one larger so it is adding to something!

enter image description here

This holds for other single digits (2-9) entered as well.

I am pretty sure it is related to how I am using the ReadConsoleA function but I have hit a bit of a wall attempting to find a solution. I have installed gdb to debug the program and assembled it as follows:

nasm -f win64 -g -o task9.obj task9.asm
GoLink /console /entry _main task9.obj kernel32.dll
gdb task9

enter image description here

But I just get the following error:

"C:\Users\Administrator\Desktop/task9.exe": not in executable format: File format not recognized

I have since read that NASM doesn't output the debug information needed for the Win64 format but I am not 100% sure about that. I am fairly sure I have the 64-bit version of GDB installed:

enter image description here

My program is as follows:

extern ExitProcess                          ;windows API function to exit process
extern WriteConsoleA                        ;windows API function to write to the console window (ANSI version)
extern ReadConsoleA                         ;windows API function to read from the console window (ANSI version)
extern GetStdHandle                         ;windows API to get the for the console handle for input/output

section .data                               ;the .data section is where variables and constants are defined

STD_OUTPUT_HANDLE   equ -11
STD_INPUT_HANDLE    equ -10

digits      db      '0123456789'            ;list of digits

input_message db   'Please enter your next number: '
length equ $-input_message

section .bss                                ;the .bss section is where space is reserved for additional variables

input_buffer:   resb 2                      ;reserve 64 bits for user input

char_written:   resb    4
chars:   resb 1                             ;reversed for use with write operation

section .text                               ;the .text section is where the program code goes

global _main                                ;tells the machine which label to start program execution from

_num_to_str:
        cmp rax, 0                          ;compare value in rax to 0
        jne .convert                        ;if not equal then jump to label
        jmp .output

.convert:
        ;get next digit value
        inc r15                             ;increment the counter for next digit

        mov rcx, 10
        xor rdx, rdx                        ;clear previous remainder result
        div rcx                             ;divide value in rax by value in rcx
                                            ;quotient (result) stored in rax
                                            ;remainder stored in rdx

        push rdx                            ;store remainder on the stack

        jmp _num_to_str

.output:
        pop rdx                             ;get the last digit from the stack

        ;convert digit value to ascii character
        mov r10, digits                     ;load the address of the digits into rsi
        add r10, rdx                        ;get the character of the digits string to display

        mov rdx, r10                        ;digit to print
        mov r8, 1                           ;one byte to be output

        call _print

        ;decide whether to loop
        dec r15                             ;reduce remaining digits (having printed one)
        cmp r15, 0                          ;are there digits left to print?
        jne .output                          ;if not equal then jump to label output

        ret

_print:
        ;get the output handle
        mov rcx, STD_OUTPUT_HANDLE          ;specifies that the output handle is required
        call GetStdHandle                   ;returns value for handle to rax

        mov rcx, rax
        mov r9, char_written

        call WriteConsoleA

        ret

_read:
        ;get the input handle
        mov rcx, STD_INPUT_HANDLE           ;specifies that the input handle is required
        call GetStdHandle

        ;get value from keyboard
        mov rcx, rax                        ;place the handle for operation

        mov rdx, input_buffer               ;set name to receive input from keyboard

        mov r8, 2                           ;max number of characters to read
        mov r9, chars                       ;stores the number of characters actually read

        call ReadConsoleA


        movzx r12, byte[input_buffer]

        ret

_get_value:
        mov rdx, input_message              ;move the input message into rdx for function call
        mov r8, length                      ;load the length of the message for function call

        call _print
        xor r8, r8
        xor r9, r9
        call _read
.end:
        ret

_main:
        mov r13, 0                          ;counter for values input
        mov r14, 0                          ;total for calculation
.loop:
        xor r12, r12
        call _get_value                     ;get value from user

        sub r12, '0'                        ;convert char to integer
        add r14, r12                        ;add value to total

        ;decide whether to loop for another character or not
        inc r13
        cmp r13, 2
        jne .loop

        ;convert total to ASCII value

        mov rax, r14                             ;num_to_str expects total in rax

        mov r15, 0                               ;num_to_str uses r15 as a counter - must be initialised
        call _num_to_str

        ;exit the program
        mov rcx, 0                          ;exit code
        call ExitProcess

I would really appreciate any assistance you can offer either with resolving the issue or how to resolve the issue with gdb.

Jester · Accepted Answer

I found the following issues with your code:

Microsoft x86-64 convention mandates rsp be 16 byte aligned.
You must reserve space for the arguments on the stack, even if you pass them in registers.
Your chars variable needs 4 bytes not 1.
ReadConsole expects 5 arguments.
You should read 3 bytes because ReadConsole returns CR LF. Or you could just ignore leading whitespace.
Your _num_to_str is broken if the input is 0.

NASM ReadConsoleA or WriteConsoleA Buffer Debugging Issue

Answers (2)

Related Questions