CODEWITHSUNDEEP
c++winapidirectwritesetwindowlong
Emre Tekmen
Emre Tekmen

Reputation: 33

SetWindowLongPtr Fails but GetLastError and errno are both 0

I am a novice programmer fiddling with the Windows API and attempting to follow the DirectWrite Documentation (probably shooting myself in the foot with that decision). I've entered a rabbit hole of debugging and have become stumped by this SetWindowLongPtr() function. Here is a snippet of what I'm working with:

LRESULT CALLBACK SimpleText::WndProc(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
    if (message == WM_NCCREATE) // the sample used WM_CREATE, which doesn't even get sent by CreateWindow(). Don't know why.
    {
        LPCREATESTRUCT pcs = (LPCREATESTRUCT)lParam; // pcs gets set successfully
        SimpleText* pSimpleText_ = (SimpleText*)pcs->lpCreateParams; // the object is initialized correctly here.

        BOOL valid = IsWindowHandleValid(hwnd); //this is indeed a valid handle.
        
        SetLastError(0); //This line is supposed to guarantee GetLastError() returns properly.
        BOOL setSuccess = ::SetWindowLongPtr( // this function returns 0
            hwnd,
            GWLP_USERDATA,
            PtrToUlong(pSimpleText_)
        );

        if (!setSuccess) {
            DWORD dwError = GetLastError(); // GetLastError() is 0. errno also 0 when tested.
        }
        return 1;
    }
    //After WM_NCCREATE is called, CreateWindow() sends NCCALCSIZE which is unhandled by the example, then it calls WM_SIZE. 
    SimpleText* pSimpleText = reinterpret_cast<SimpleText*>(
        ::GetWindowLongPtr(hwnd, GWLP_USERDATA)); // Undefined data here.

    if (pSimpleText) //passes nullcheck
    {
        switch (message)
        {
        case WM_SIZE:
        {
            UINT width = LOWORD(lParam);
            UINT height = HIWORD(lParam);
            if(pSimpleText->pRT_ != nullptr) // read access violation, uninitialized object.
                pSimpleText->OnResize(width, height);
        }
        //other cases...
        }
    }
    return DefWindowProc(
        hwnd,
        message,
        wParam,
        lParam
    );
}

Windows Documentation makes a note about 32 and 64-bit, which might be a problem. It also says the following:

The SetWindowLongPtr function fails if the process that owns the window specified by the hWnd parameter is at a higher process privilege in the UIPI hierarchy than the process the calling thread resides in.

This is above my understanding, but may also be related.

Pastebin Link to SimpleText.cpp if you need more context that I may have missed in this post.

Upvotes: 0

Views: 95

Answers (0)

Related Questions