CODEWITHSUNDEEP
c++windowsbcryptmingw-w64
Evan Hendler
Evan Hendler

Reputation: 330

BCryptGenRandom STATUS_INVALID_HANDLE

I am trying to generate cryptographically secure numbers in Windows. I have the following code:

#ifndef w64crypt
#define w64crypt
#include <windows.h>
#include <bcrypt.h>
unsigned long getSeed(ULONG buffer_size){ 
    NTSTATUS status=0;
    ULONG flags=0;
    PUCHAR c_seed;
    BCRYPT_ALG_HANDLE handle;
    status=BCryptOpenAlgorithmProvider(
            &handle,BCRYPT_RNG_ALGORITHM,NULL,0
    );
    if(!BCRYPT_SUCCESS(status)){
            cout << "BCryptOpenAlgorithmProvider";
            printf("%X",status);
    }
    status=BCryptGenRandom(
            handle,c_seed,128,0
    );
    if(!BCRYPT_SUCCESS(status)){
            cout << "Error in BCryptGenRandom";
            printf("%X",status);
    }
    status=BCryptCloseAlgorithmProvider(
            handle,0
    );
    if(!BCRYPT_SUCCESS(status)){
            cout << "BCryptCloseAlgorithmProvider";
            printf("%X",status);
    }
    unsigned long seedNo=(unsigned long)c_seed;
    return seedNo;
}
#endif

When I run this, I get Error in BCryptGenRandom C0000008, which is STATUS_INVALID_HANDLE. This is what the documentation calls for:

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptopenalgorithmprovider

NTSTATUS BCryptOpenAlgorithmProvider(
  BCRYPT_ALG_HANDLE *phAlgorithm,
  LPCWSTR           pszAlgId,
  LPCWSTR           pszImplementation,
  ULONG             dwFlags
);

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom

NTSTATUS BCryptGenRandom(
  BCRYPT_ALG_HANDLE hAlgorithm,
  PUCHAR            pbBuffer,
  ULONG             cbBuffer,
  ULONG             dwFlags
);

The above link also states that pszAlgId must be:

The handle of an algorithm provider created by using the BCryptOpenAlgorithmProvider function. The algorithm that was specified when the provider was created must support the random number generator interface.

Looking at https://learn.microsoft.com/en-us/windows/win32/seccng/cng-algorithm-identifiers, BCRYPT_RNG_ALGORITHM is my ideal candidate.

I'm compiling this using mingw-w64, cross-compiling from Ubuntu 18.04.

Why am I receiving this NTSTATUS?

Upvotes: 3

Views: 3078

Answers (2)

Evan Hendler
Evan Hendler

Reputation: 330

This solved it for me:

unsigned long getSeed(int size){
    BCRYPT_ALG_HANDLE handle;
    int Buffer;
    PUCHAR bPointer=(PUCHAR)&Buffer;
    if(!BCRYPT_SUCCESS(BCryptOpenAlgorithmProvider(
            &handle,BCRYPT_RNG_ALGORITHM,NULL,0
    ))){
            std::cout << "BCryptOpenAlgorithmProvider failure\n";
    }
    NTSTATUS status=BCRYPT_SUCCESS(BCryptGenRandom(
            &handle,bPointer,sizeof(Buffer),0
    ));
    if(!BCRYPT_SUCCESS(status)){
            printf("%X",status);
            std::cout << "Error in BCryptGenRandom";
    }
    BCryptCloseAlgorithmProvider(
            handle,0
    );
    return static_cast<int>(reinterpret_cast<intptr_t>(bPointer));
}

The original code claiming that the buffer was 128 bytes in it's call to BCryptGenRandom, which wasn't true. The buffer in the original code called c_seed was a wild pointer, most likely with size 8.

In this case, c_seed, now aliased as bPointer, is no longer wild - it is pointing to int Buffer. The size of this buffer is being passed to the function using the sizeof macro, so though likely that it is size 8, it can accommodate whatever target architecture the cross-compiler I was using needs.

Upvotes: 2

rdfr
rdfr

Reputation: 56

I know that I'm late, but hopefully this helps people who're having the same issue.

The documentation doesn't mention it, but you can use one of the predefined pseudo-handles for common algorithms. Here's an example

unsigned int GenAESKey(char* keyb) {
NTSTATUS s;
/*BCRYPT_ALG_HANDLE algh;
s = BCryptOpenAlgorithmProvider(&algh, BCRYPT_RNG_ALGORITHM, MS_PLATFORM_CRYPTO_PROVIDER, 0);
if (s != STATUS_SUCCESS)
    return (unsigned int)s;*/
s = BCryptGenRandom(BCRYPT_RNG_ALG_HANDLE, (PUCHAR)keyb, CABINET_AES_KEY_S, 0);
if (s != STATUS_SUCCESS)
    return (unsigned int)s;
return 0;

You can see the list of all the predefined pseudo-handles here

Upvotes: 4

Related Questions