CODEWITHSUNDEEP
c#structpinvokemarshalling
Aster Veigas
Aster Veigas

Reputation: 876

Marshalling a struct from pointer in x64

I have three struct's:

    [StructLayout(LayoutKind.Sequential)]
    internal struct COPYDATASTRUCT
    {
        public IntPtr dwData;       // Specifies data to be passed
        public int cbData;          // Specifies the data size in bytes
        public IntPtr lpData;       // Pointer to data to be passed
    }

    public struct SHELLTRAYDATA
    {
        public UInt32 dwUnknown;
        public UInt32 dwMessage;
        public NID_XX nid;
    }

    public struct NID_XX
    {
        public UInt32 cbSize;
        public IntPtr hWnd;
        public uint uID;
        public uint uFlags;
        public uint uCallbackMessage;
        public IntPtr hIcon;
    }

In my WndProc I do the following:

      case WM_COPYDATA:
                {
                    COPYDATASTRUCT cp = (COPYDATASTRUCT)Marshal.PtrToStructure(lParam, typeof(COPYDATASTRUCT));
                    if(cp.dwData == SH_TRAY_DATA)
                    {
                        var shellTrayData = (SHELLTRAYDATA)Marshal.PtrToStructure(cp.lpData,typeof(SHELLTRAYDATA));
                        HandleNotification(shellTrayData);
                    }
                }

When my application is running on x86 it works fine. When I run it on x64 I don't get an hIcon and moreover the hWnd is invalid. When I target the application to x86 and run on x64 it works fine though. I know the problem lies in Marshalling. Do I have to manually Marshal the structure? Need help on this. I would prefer to have the same struct for both x64 and x86

EDIT:

The Unmanaged structures are as follows:

typedef struct tagCOPYDATASTRUCT 
{
    ULONG_PTR dwData;
    DWORD cbData;
    _Field_size_bytes_(cbData) PVOID lpData;
} COPYDATASTRUCT, *PCOPYDATASTRUCT;

// data sent by shell via Shell_NotifyIcon
typedef struct _SHELLTRAYDATA
{
    DWORD dwUnknown;
    DWORD dwMessage;
    NID_XX nid;
} *PSHELLTRAYDATA;

// sub structure common to all others


  typedef struct
  {
  DWORD cbSize;
  HWND hWnd;
  UINT uID;
  UINT uFlags;
  UINT uCallbackMessage;
  HICON hIcon;
  } NID_XX, *PNID_XX;
  typedef const NID_XX * PCNID_XX;

EDIT: Sizes of the structs are as follows:

Unmanaged:

  1. COPYDATASTRUCT: 12(X86) and 24(x64)
  2. SHELLTRAYDATA: 32(X86) and 48(X64)
  3. NID_XX: 24(X86) and 40(X64)

Managed:

  1. COPYDATASTRUCT: 12(X86) and 24(x64)
  2. SHELLTRAYDATA: 32(X86) and 48(X64)
  3. NID_XX: 24(X86) and 40(X64)

It's the same on both the sides.

Upvotes: 1

Views: 1992

Answers (2)

Aster Veigas
Aster Veigas

Reputation: 876

Since Handles are 64bits on 64bit builds, while the structure we receive still contain 32bits. Changing IntPtr to Uint32 for hIcon and hWnd did the trick. Now this struct works on both x64 and x86 architectures. I always knew serialization into JSON was definitely deviating from my requirement. Thanks for all the help.

Upvotes: 1

David Heffernan
David Heffernan

Reputation: 613562

These structs vary on different architectures. The IntPtr values are 32 bits wide, or 64 bits wide depending on the architecture. That has obvious consequences for the size of the members, and perhaps less obvious consequences for alignment.

There's no way that you can use the same struct irrespective of architecture. You need to be aware of the architecture of both processes. Whenever sender and receiver don't have the same architecture, then you are in trouble.

You could pick a fixed layout, using C# long instead of IntPtr. Types like HWND and HICON will always fit in a C# long. Although do beware of handle types that only have meaning in a particular process. For instance, HICON values won't have meaning once you send them to a different process.

Instead of using a fixed layout struct I think I would serialize. Serialize the data that you wish to send to JSON, for example. This can readily be packed up into a string and sent via WM_COPYDATA.

Upvotes: 0

Related Questions