CODEWITHSUNDEEP
c++windows
Fred
Fred

Reputation: 347

Getting address of caller in c++

At the moment I'm working on a anticheat. I added a way to detect any hooking to the directx functions, since those are what most cheats do.

The problem comes in when a lot of programs, such as OBS, Fraps and many other programs that hook directx get their hook detected too.

So to be able to hook directx, you will most probabbly have to call VirtualProtect. If I could determine what address this is being called from, then I could loop through all dll's in memory, and then find what module it has been called from, and then sending the information to the server, maybe perhaps even taking a md5 hash and sending it to the server for validation.

I could also hook the DirectX functions that the cheats hook and check where those get called from (since most of them use ms detours).

I looked it up, and apparently you can check the call stack, but every example I found did not seem to help me.

Upvotes: 0

Views: 2837

Answers (2)

Mike Vine
Mike Vine

Reputation: 9837

Ray Chen's blog 'The old new thing' covers using return address' to make security decisions and why its a pretty pointless thing

https://devblogs.microsoft.com/oldnewthing/20060203-00/?p=32403

https://devblogs.microsoft.com/oldnewthing/20040101-00/?p=41223

Basically its pretty easy to fake (by injecting code or using a manually constructed fake stack to trick you). Its Windows centric but the basic concepts are generally applicable.

Upvotes: 0

Basile Starynkevitch
Basile Starynkevitch

Reputation: 1

This -getting the caller's address- is not possible in standard C++. And many C++ compilers might optimize some calls (e.g. by inlining them, even when you don't specify inline, or because there is no more any framepointer, e.g. compiler option -fomit-frame-pointerfor x86 32 bits with GCC, or by optimizing a tail-call ....) to the point that the question might not make any sense.

With some implementations and some C or C++ standard libraries and some (but not all) compiler options (in particular, don't ask the compiler to optimize too much*) you might get it, e.g. (on Linux) use backtrace from GNU glibc or I.Taylor's libbacktrace (from inside GCC implementation) or GCC return address builtins.

I don't know how difficult would it be to port these to Windows (Perhaps Cygwin did it). The GCC builtins might somehow work, if you don't optimize too much.

Read also about continuations. See also this answer to a related question.

Note *: on Linux, better compile all the code (including external libraries!) with at most g++ -Wall -g -O1 : you don't want too much optimization, and you want the debug information (in particular for libbacktrace)

Upvotes: 1

Related Questions