CODEWITHSUNDEEP
c++
99Boboster99
99Boboster99

Reputation: 63

Converting working x64 solution to x86 (VS2017)

I have a VS2017 solution (unmanaged API hook) that is working perfectly when built for x64 (Debug or Release). I am trying to the same under x86. I have no compile or link errors, but my application crashes with an ntdll.dll exception.

I used Deviare's SpyStudio to monitor which files are being opened before my app crashes. My x86 app (when hooked) is trying to access the "System32" directory where (I think) it should be going to "SysWOW64" or "winsxs" directories.

Here is the Deviare SpyStudio screenshot of the x86 application running, without the hook activated, it works fine;

https://imagizer.imageshack.com/img924/7403/k49M3x.png

This is the SpyStudio screenshot with the hook activated;

https://imagizer.imageshack.com/img923/9698/lYLtLJ.png

It looks like my "ntdll.dll" library is not being selected properly by VS2017, I thought this is done automatically, since in my code I am using this;


#pragma comment (lib, "ntdll") 

<....>
HMODULE hDll_ntdll = LoadLibrary(TEXT("ntdll.dll"));
<....>

I did notice that when I comment out the #pragma comment (lib, "ntdll") line above I get "unresolved external" errors.

What am I doing wrong or what did I forget to do?

Upvotes: 1

Views: 8779

Answers (1)

99Boboster99
99Boboster99

Reputation: 63

I figured it out! :) For anyone else running into this problem, it is the x86 calling convention that caused my errors. Simply adding "__stdcall" to all my function definitions did the trick. Example: for x64 and for x86 I changed this function call, that only worked in x64;

NTSTATUS NtCreateFile_Hook(OUT PHANDLE FileHandle, IN ACCESS_MASK DesiredAccess, IN POBJECT_ATTRIBUTES ObjectAttributes, OUT PIO_STATUS_BLOCK IoStatusBlock, IN PLARGE_INTEGER AllocationSize OPTIONAL, IN ULONG FileAttributes, IN ULONG ShareAccess, IN ULONG CreateDisposition, IN ULONG CreateOptions, IN PVOID EaBuffer OPTIONAL, IN ULONG EaLength)

To this, which works for both x64 and x86;

NTSTATUS __stdcall NtCreateFile_Hook(OUT PHANDLE FileHandle, IN ACCESS_MASK DesiredAccess, IN POBJECT_ATTRIBUTES ObjectAttributes, OUT PIO_STATUS_BLOCK IoStatusBlock, IN PLARGE_INTEGER AllocationSize OPTIONAL, IN ULONG FileAttributes, IN ULONG ShareAccess, IN ULONG CreateDisposition, IN ULONG CreateOptions, IN PVOID EaBuffer OPTIONAL, IN ULONG EaLength)

This works perfectly! Hopefully, there aren't any drawbacks adding "__stdcall" to the x64 platform. So far, it works properly.

Upvotes: 1

Related Questions