CODEWITHSUNDEEP
.netvb.netmarshallingnative-code
LCJ
LCJ

Reputation: 22661

Why long datatype work for this marshalling?

My knowlege beyond .Net is very limited - but I put a good lot of time reading through many related articles.

HCRYPTPROV datatype documentaion says it is of type ULONG_PTR.

The following references suggest use of IntPtr corresponding to this.

  1. Is there a definitive guide to cross platform (x86 and x64) PInvoke and windows data types?
  2. Using MS crypto library on server 2012 - CryptCreateHash error code 87: ERROR_INVALID_PARAMETER
  3. Calling AuditQuerySystemPolicy() (advapi32.dll) from C# returns "The parameter is incorrect"

But, in the following code I am using long datatype and it works just fine. Is there any scenario it will give incorrect result? What is the reason it is working with long ?

Framework: .Net 2.0;
Architecture: 64 Bit;
OS: Windows Server 2012 R2;
Visual Studio: 2013

CODE

Module Module1

    Private Declare Function CryptAcquireContext Lib "advapi32.dll" _
  Alias "CryptAcquireContextA" ( _
ByRef phProv As Long, ByVal pszContainer As String, ByVal pszProvider As String, _
  ByVal dwProvType As Integer, ByVal dwFlags As Integer) As Integer

    Private Declare Function CryptCreateHash Lib "advapi32.dll" (ByVal hProv As Long, _
        ByVal Algid As Integer, ByVal hKey As Integer, ByVal dwFlags As Integer, _
    ByRef phHash As Integer) As Integer

    Private Declare Function GetLastError Lib "kernel32" () As Integer

    Sub Main()

        Dim sClearText As String
        sClearText = "test1"

        Dim lHCryptprov As Long
        Dim sProvider As String
        Const MS_DEF_PROV As String = "Microsoft Base Cryptographic Provider v1.0"

        Dim lHHash As Integer
        Dim sInputBuffer As String

        Const ALG_CLASS_HASH As Integer = 32768
        Const ALG_TYPE_ANY As Integer = 0
        Const ALG_SID_MD5 As Integer = 3
        Const PROV_RSA_FULL As Integer = 1
        Const CRYPT_MACHINE_KEYSET As Integer = &H20
        Const CALG_MD5 As Integer = ((ALG_CLASS_HASH Or ALG_TYPE_ANY) Or ALG_SID_MD5)


        sInputBuffer = sClearText
        'Get handle to the default CSP
        sProvider = MS_DEF_PROV & vbNullChar


        Dim errorCode As Integer
        Dim r As Long
        r = CryptAcquireContext(lHCryptprov, "", sProvider, PROV_RSA_FULL, CRYPT_MACHINE_KEYSET)
        errorCode = GetLastError()

        Dim hashResult As Boolean
        hashResult = CBool(CryptCreateHash(lHCryptprov, CALG_MD5, 0, 0, lHHash))
        errorCode = GetLastError()


        Console.WriteLine(hashResult)
        Console.ReadLine()


    End Sub

End Module

Upvotes: 0

Views: 327

Answers (1)

Hans Passant
Hans Passant

Reputation: 942408

Architecture: 64 Bit;

That is what most likely keeps you out of trouble. ULONG_PTR is an integer type whose size depends on the bitness of the process. In a 32-bit process it is 32-bits, in a 64-bit process it is 64-bits. The most direct .NET type equivalent is UIntPtr.

Its usage here is as a handle, the actual value does not matter. You don't do any arithmetic on the value, you obtain it from CryptAcquireContext() and simply pass it on to CryptCreateHash(). So it doesn't matter whether the type you select in your pinvoke declaration is signed or unsigned. The reason that IntPtr is the more common advice, it is a [CLSCompliant] type.

A Long in VB.NET is a 64-bit signed integer type. So matches ULONG_PTR when your program runs as a 64-bit process.

You can get into trouble when it doesn't. Either because your program runs on a machine with a 32-bit operating system or when you use the default settings for a new project on VS2013. The relevant setting is Project > Properties > Compile tab > Target CPU. Change it from AnyCPU to x86 to get into trouble. Note there's also the "Prefer 32-bit" checkbox, it is disabled because you target .NET 2.0

When you start your program after making this change, you should now get a debugger break when you call CryptCreateHash(). There is a dedicated Managed Debugging Assistant named PInvokeStackImbalance that should notice that the stack is out of whack by 4 when the function returns. And invokes a debugger break to tell you about it. A stack imbalance is a very nasty problem that can cause arbitrary program failure, including and not limited to the problem this site is named for, especially in the Release build. Very hard to debug without that MDA helping.

No point in getting this wrong intentionally, use IntPtr instead so it matches the native type. Even more correct is when you declare it HandleRef instead. That's a wrapper that ensures that the handle cannot be destroyed while the native code is busy executing and should be used if you have a Finalize() method that calls CryptReleaseContext(). Even more correct is to use a safe handle wrapper type, like the .NET Framework does, it has a critical finalizer that runs even if the program bombs badly. Only truly necessary if your code runs inside of an unmanaged hosting program, SQL Server as a common example.

Upvotes: 2

Related Questions