Ayyware
Ayyware

Reputation: 53

Undefined function with extern "c" and assembly

Hey so I'm getting an undefined function error when I'm trying to compile my code and I don't know why.

Here are my files: Api.h

extern "C" NTSTATUS NtWriteVirtualMem(HANDLE ProcessHandle, PVOID BaseAddress, PVOID Buffer, ULONG NumberOfBytesToWrite, PULONG NumberOfBytesWritten);

API_ASM.asm

.code
_NtWriteVirtualMem:
    mov r10, rcx
    mov eax, 3Ah
    syscall
    ret

end

Both of them compile correctly but I get an error because NtWriteVirtualMem is defined but I defined it in my asm?

EDIT 1.

So I changed my code to:

PUBLIC NtWriteVirtualMem

_TEXT SEGMENT
NtWriteVirtualMem PROC
    mov r10, rcx
    mov eax, 3Ah
    syscall
    ret
    NtWriteVirtualMem ENDP
_TEXT ENDS
END

The program now compiles but the write doesn't work. I've tested it to see if writeprocessmemory works and it does. Also, my IDE shows that NtWriteVirtualMem is still undefined when I hover over the name in the C source.

EDIT 2.

Also, the NTSTATUS return code for the operation is a negative max integer.

Upvotes: 1

Views: 861

Answers (1)

Peter Cordes
Peter Cordes

Reputation: 363980

Windows x64 doesn't use a leading underscore _ in asm symbol names. Your asm label name should just be NtWriteVirtualMem, matching the extern "C" function name.

See Agner Fog's calling convention guide, for example. http://www.agner.org/optimize/calling_conventions.pdf

(Also make sure you tell the assembler to export it, if necessary. Like with NASM you'd use global to modify that label declaration. IDK if MASM needs proc / endp or if a label can work.)


Your fixed version links properly, and you confirmed that execution goes through the syscall instruction. Probably you're passing the wrong args, or passing them the wrong way, or using the wrong call number.

The syscall ABI is undocumented and unsupported on Windows, and can change between kernel versions. https://j00ru.vexillium.org/syscalls/nt/64/ says that the call number for NtWriteVirtualMem is 3Ah on all Windows 10 versions, but I don't know if you're passing other args correctly. On earlier verisons of Windows, 0x3a is a different system call.

I'll leave this part of the question for someone who uses Windows and knows what that system call does.

Upvotes: 1

Related Questions