Implementing a keyboard hook under windows produces massive bugs

Question

I tried to implement a basic hook under windows. While it technically works (my handle gets called) it produces massive bugs, that i do not know how to fix. When I run the programm and try to write something in the firefox.exe searchbar or the explorer.exe addres bar the programs crash.

That is the code i currently tried. I stripped everything unnecessary but it still doesnt work

main.cc

#include <iostream>

#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#include <shellapi.h>


int main (int argc, char** argv){
  HINSTANCE dll = NULL;
  HOOKPROC proc = NULL;
  HHOOK hook = NULL;

  MSG msg;
  BOOL b_ret = FALSE;

  dll = LoadLibraryA("hookdll.dll");
  if (dll == NULL) {
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    return 1;
  }
  proc = (HOOKPROC)GetProcAddress(dll, "KeyboardHook");
  if (proc == NULL) {
    FreeLibrary(dll);
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    return 1;
  }
  hook = SetWindowsHookEx(WH_CALLWNDPROC, proc, dll, 0);
  if (hook == NULL) {
    FreeLibrary(dll);
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    return 1;
  }

  while (GetMessage(&msg, NULL, WH_KEYBOARD, WM_KEYLAST) > 0) {
    TranslateMessage(&msg);
    DispatchMessage(&msg);
  }
  
  UnhookWindowsHookEx(hook);
  FreeLibrary(dll);

  return 0;
}

And in the DLL:

hooks.h

#ifndef __LOG_DLL_HOOKS_H__
#define __LOG_DLL_HOOKS_H__

#include <iostream>
#include <fstream>

#define WIN32_LEAN_AND_MEAN
#include <Windows.h>

extern "C" LRESULT CALLBACK KeyboardHook(
  _In_ int    code,
  _In_ WPARAM wParam,
  _In_ LPARAM lParam);

#endif // !__LOG_DLL_HOOKS_H__

hooks.cc

#include "Hooks.h" 

extern "C" LRESULT CALLBACK  KeyboardHook(
  _In_ int    code,
  _In_ WPARAM wParam,
  _In_ LPARAM lParam) {

  if (code == HC_ACTION) {

  }
  return CallNextHookEx(NULL, code, wParam, lParam);
}

As you can see it barely does anything. I worked with the official windows docs and did everything the right way (or so i thought) The only thing that could make a difference is the return value of KeyboardHook, but https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/ms644984(v=vs.85) states that:

If code is less than zero, the hook procedure must return the value returned by CallNextHookEx.

If code is greater than or equal to zero, and the hook procedure did not process the message, it is highly recommended that you call CallNextHookEx and return the value it returns;

I read that as: "Return CallNextHookEx either way", which i did.

Thanks for any answeres

Answer 1

Well, for starters, why are you using a keyboard function for a window procedure hook? I think you meant to use WH_KEYBOARD instead of WH_CALLWNDPROC when calling SetWindowsHookEx().

Also, your GetMessage() call is wrong, because WH_KEYBOARD is not a message identifier. You would need to use WM_KEYFIRST instead, to match your use of WM_KEYLAST (since you are clearly only interested in dispatching keyboard messages).

However, you are setting the dwThreadId parameter of SetWindowsHookEx() to 0, which means you are hooking not just your own thread but all threads of all processes globally. A WH_KEYBOARD hook runs in the context of the thread that installs it, which means internally the OS will have to delegate the hooked keyboard messages of those other threads to your thread, and it does that by sending messages to your thread. But, your message loop is not going to be processing any of those messages because it is filtering for only keyboard messages (of which it will never receive, since your thread has no UI of its own).

With all of that said, try this instead:

#include <iostream>

#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#include <shellapi.h>


int main (){
  HINSTANCE dll = LoadLibraryA("hookdll.dll");
  if (dll == NULL) {
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    return 1;
  }

  HOOKPROC proc = (HOOKPROC) GetProcAddress(dll, "KeyboardHook");
  if (proc == NULL) {
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    FreeLibrary(dll);
    return 1;
  }

  HHOOK hook = SetWindowsHookEx(WH_KEYBOARD, proc, dll, 0);
  if (hook == NULL) {
    std::cerr << "WinAPI Error: " << GetLastError() << "\n";
    FreeLibrary(dll);
    return 1;
  }

  MSG msg;
  while (GetMessage(&msg, NULL, 0, 0) > 0) {
    TranslateMessage(&msg);
    DispatchMessage(&msg);
  }
  
  UnhookWindowsHookEx(hook);
  FreeLibrary(dll);

  return 0;
}