CODEWITHSUNDEEP
winapiassemblystructx86masm
William
William

Reputation: 223

How to pass a struct by value in x86 assembly

I'm trying to call a function from the windows api in masm.

This is the signature:

BOOL WINAPI SetConsoleScreenBufferSize(
  _In_ HANDLE hConsoleOutput,
  _In_ COORD  dwSize
);

The COORD structure dwSize is passed by value, but when I try to call it the function fails.

Looks like this:

                .DATA
dwSize          COORD   <20, 20>
                .CODE
                INVOKE  SetConsoleScreenBufferSize,
                        hConsoleOutput,
                        dwSize

This causes a type error and the program won't assemble. If I pass a reference to the struct, the program assembles but the function does not work. I've tried with other functions that accept structs by value, with no success.

Upvotes: 1

Views: 862

Answers (2)

Anders
Anders

Reputation: 101606

COORD is just two 16-bit numbers packed together and passed as normal 32-bit number.

MSVC (x86) turns

COORD cord = { 0x666, 0x42 };
SetConsoleScreenBufferSize(0, cord);

into 

33db            xor     ebx,ebx

66c745986606    mov     word ptr [ebp-68h],666h    ; store cord.x
66c7459a4200    mov     word ptr [ebp-66h],42h     ; store cord.y

ff7598          push    dword ptr [ebp-68h]        ; push the whole struct
53              push    ebx                        ; push 0
ff1540104000    call    dword ptr [image00400000+0x1040 (00401040)] ; SetConsoleScreenBufferSize

After push'ing but before the call the stack starts with:

00000000 00420666 ...

xor-zeroing a register and then pushing that is a missed optimization vs. push 0 of an immediate zero. Storing to the stack first is also only because the source was compiled with optimization disabled.

Upvotes: 2

Michael Petch
Michael Petch

Reputation: 47573

Hans is correct Invoke doesn't understand how to pass a struct by value. COORD is 2 16-bit values which happens to be the size of a DWORD. In the case of COORD you can cast it to a DWORD as a parameter to Invoke . This should work:

                .DATA
dwSize          COORD   <20, 20>
                .CODE
                INVOKE  SetConsoleScreenBufferSize,
                        hConsoleOutput,
                        DWORD PTR [dwSize]

Note: It is important to understand that since COORD happened to be the size of a DWORD we could get away with this. For structures that don't have a size that can be pushed on the stack directly you'd have to build the structure on the stack and use the CALL instruction rather than Invoke.

Upvotes: 3

Related Questions