Globally installed keyboard hook prevents keyboard input to other applications

Question

I am setting a global hook for keyboard. When I give keyboard inputs to other applications, the application does not receive the input and it hangs. When the console is stopped, the application recovers and the keyboard inputs are posted together.

DLL source:

#include <iostream>
#include <Windows.h>
#include <string>
using namespace std;
#define DLLEXPORT __declspec(dllexport)

DLLEXPORT bool installhook();
DLLEXPORT void unhook();
DLLEXPORT string TestLoaded();
DLLEXPORT LRESULT CALLBACK KeyboardProc ( int code, WPARAM wParam, LPARAM lParam );

static HHOOK kb_hook;
string test = "not loaded";
HINSTANCE hDLL;

DLLEXPORT LRESULT CALLBACK KeyboardProc ( int code, WPARAM wParam, LPARAM lParam )
{
    if(code == HC_ACTION)   // if there is an incoming action and a key was pressed
    {
        switch(wParam)
        {
        case VK_SPACE:
            printf("Space was pressed\n"); //tried without this also
            MessageBoxA(NULL, "Hi", "Space", MB_OK);
            break;
        }
    }
    return CallNextHookEx(NULL, code, wParam, lParam);
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
    test = "loaded";
    switch(ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        hDLL = hModule;
        break;
    }
    printf("test str = %s \n", test.c_str());
    return TRUE;
}

bool installhook()
{
    kb_hook = SetWindowsHookEx(WH_KEYBOARD, KeyboardProc, hDLL, NULL);
    if(!kb_hook)
    {
        return false;
    }
    return true;
}

void unhook()
{
    if(kb_hook)
    {
        UnhookWindowsHookEx(kb_hook);
    }
}

string TestLoaded()
{
    return test;
}

Console applicatioon source:

#include <iostream>
#include <Windows.h>
#include <string>
#define DLLIMPORT __declspec(dllimport)
using namespace std;

DLLIMPORT void unhook();
DLLIMPORT bool installhook();
DLLIMPORT string TestLoaded();

int main()
{
    cout << TestLoaded() <<endl;
    installhook();
    for(int i = 1; i<=10 ; i++)
    {
        //Do some keyboard activities in this 10 secs
        Sleep(1000);
        cout << i<<endl;
    }

    unhook();
    cin.get();
    return 1;
}

My suspicion was that since the dll will be loaded into each process in the process's own address space and console would not be present in other applications, it gets void and crashed. So I removed the console outputs and replaced with messagebox. Then also no difference.

What could be the problem?

Update:

I tried to do a local hook to a specific thread before trying it global. But I get Parameter is incorrect error 87 at setwindowshookex. Below are the updated code:

dll:

bool installhook(DWORD ThreadId) //exporting this function
{
    kb_hook = SetWindowsHookEx(WH_KEYBOARD, KeyboardProc, NULL, ThreadId); //tried with the dll module's handle also instead of NULL
    if(!kb_hook)
    {
        printf("SetWindowsHookEx failed : %d\n", GetLastError());
        return false;
    }
    return true;
}

Console application source:

DWORD myThread()
{
    cout<< "Thread started\n";
    char str[250];
    cin>>str;
    return 0;
}

int main()
{
    cout << TestLoaded() <<endl;
    DWORD myThreadID;
    HANDLE myHandle = CreateThread(0, 0, (LPTHREAD_START_ROUTINE)myThread, NULL, 0, &myThreadID);
    installhook(myThreadID);
    for(int i = 0; i<100 ; i++)
    {
        Sleep(100);
        if(i%10 == 0)
        {
            cout << i<<endl;
        }
    }
    unhook();
}

Answer 1

Thanks for all the inputs in answers and comments.

I have found out the actual problem. The mistake I made was trying to use console window without any message queue.

If I understand correctly, console windows are hosted by conhost.exe and they don't have any message pumps. And the hook works correctly only if the application which installs it has a message queue (should explore more on why it's this way). See below for ways you can make it work

If you are not posting any message to the console application:

Replace the for loop in the console application's main with this:

MSG msg;
while(GetMessage(&msg, NULL, 0, 0) > 0)
{
        TranslateMessage(&msg);
        DispatchMessage(&msg);
}

In case you are posting any message to the console application:

Create a window using CreateWindowEx, there is an option for a message only window also. You would have to create a class and assign a CALLBACK process. Read here for more details. Create that and pass the handle along to the hook dll and postmessage to the handle. Use the loop for Getting msg and dispatching it (mentioned above). Then all the messages you post the dummy window from your hook dll can be processed using the CALLBACK window process.

References:

Why must SetWindowsHookEx be used with a windows message queue

CreateWindowEx MSDN

Answer 2

Try to use WH_KEYBOARD_LL. You can set global hook even without dll declaring hook function in you process. Plus, you should detect space action using PKBDLLHOOKSTRUCT struct

LRESULT CALLBACK KeyboardProc ( int code, WPARAM wParam, LPARAM lParam )
{
   if ( code == HC_ACTION )  
   {
       switch ( wParam )
       {
          case WM_KEYDOWN:
          {
              // Get hook struct
              PKBDLLHOOKSTRUCT p = ( PKBDLLHOOKSTRUCT ) lParam;
              if ( p->vkCode == VK_SPACE)
              {
                MessageBoxA( NULL, "Hi", "Space", MB_OK );
              }
          }
       break;
       }
   }
   return CallNextHookEx( NULL, code, wParam, lParam );
 }

 ....
 // Somewhere in code
 kb_hook = SetWindowsHookEx( WH_KEYBOARD_LL, KeyboardProc, NULL, NULL );

Answer 3

I had the same issue, working with QT, the GUI would be blocked (as planned) but whenever it came back online, it would process my keyboard and mouse clicks.

I am not sure if this is the most efficient way of handling it, but to solve this, I handled all the keyboard and mouse events separately. If, some task was in progress, I would just ignore the key event.

Otherwise I guess it just queues up and waits for its' turn!