CODEWITHSUNDEEP
c++encryptionopenssl
Siler
Siler

Reputation: 9484

OpenSSL API with AES encryption with GCM (Galois Counter Mode)

I'm using the OpenSSL API in C++ to perform basic encryption/decryption of private keys. I was experimenting with different encryption modes, and for some reason I get an error when using GCM (Galois Counter Mode) encryption with AES. Other modes, such as CBC, work fine.

The relevant code (edited for clarity) is:

void encrypt_private_key(
  EVP_PKEY* key_handle,
  const char* password, 
  std::size_t password_length
)
{
  std::unique_ptr<BIO, decltype(&BIO_free)> bio_ptr(BIO_new(BIO_s_mem()), &BIO_free);
  if (!bio_ptr) { /* handle error */ }

  const EVP_CIPHER* enc = EVP_aes_256_gcm();
  int result = PEM_write_bio_PKCS8PrivateKey(
    bio_ptr.get(), 
    key_handle, 
    enc, 
    const_cast<char*>(password),
    password_length, 
    nullptr, 
    nullptr
  );

  if (result != 1) { /* handle error code */ }

  // ... do stuff with encrypted key
}

Here, I get an error code. When I print the error code using ERR_reason_error_string(), I get the message "error setting cipher params"

Note that if I change the cipher from EVP_aes_256_gcm() to EVP_aes_256_cbc(), it works fine.

Reading the relevant docs, I see that GCM mode can take some extra parameters and tuning using the EVP_CIPHER_CTX_ctrl function. However, AFAIK this seems to be optional. I see no documented reason why EVP_aes_256_gcm() shouldn't work normally with no additional parameters being set, apart from what is passed to PEM_write_bio_PKCS8PrivateKey.

So what am I doing wrong here?

Why do I get an error when trying to use GCM mode with AES encryption? Could it be OpenSSL simply doesn't implement this? (I'm using an older version of OpenSSL here - OpenSSL 1.0 - so this could be the issue. However, if it didn't support GCM mode I would expect to get a compile time error telling me that EVP_aes_256_gcm() is not even defined, rather than a run-time error like this.)

Upvotes: 3

Views: 2655

Answers (2)

dgrandm
dgrandm

Reputation: 383

I suggest trying BoringSSL library, is a fork of OpenSSL branch by Google. Apparently they use AES 265-bit in GCM for encrypting cookies starting Chrome version 80 on Windows (Feb, 2020)

Upvotes: -2

Reinier Torenbeek
Reinier Torenbeek

Reputation: 17383

This GCM mode of operation is not supported as a cipher to encrypt the private key. With the openssl genpkey tool, version 1.0.2n, for generating PKCS#8-formatted private keys, the following command shows that same error message that you encounter:

$ openssl genpkey -algorithm rsa -aes-128-gcm
..............++++++
.......++++++
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
Error writing key
140736195879880:error:0D0A7072:asn1 encoding routines:PKCS5_pbe2_set_iv:error setting cipher params:p5_pbev2.c:131:
140736195879880:error:2307D00D:PKCS12 routines:PKCS8_encrypt:ASN1 lib:p12_p8e.c:86:

Trying the same thing with 1.1.1 looks a little cleaner, but still no luck:

$openssl genpkey -algorithm rsa -aes-128-gcm
genpkey: cipher mode not supported

For that version, the code for the genpkey tool at the main program level explicitly tests the provided cipher here to check for unsupported modes:

        if (EVP_CIPHER_mode(cipher) == EVP_CIPH_GCM_MODE ||
            EVP_CIPHER_mode(cipher) == EVP_CIPH_CCM_MODE ||
            EVP_CIPHER_mode(cipher) == EVP_CIPH_XTS_MODE ||
            EVP_CIPHER_mode(cipher) == EVP_CIPH_OCB_MODE) {
            BIO_printf(bio_err, "%s: cipher mode not supported\n", prog);

Indeed, for verification purposes, this command works for CBC mode of operation:

$ openssl genpkey -algorithm rsa -aes-128-cbc
.................++++++
..........................................................++++++
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIICzzBJBgkqhkiG9w0BBQ0wPDAbBgkqhkiG9w0BBQwwDgQI/XYjzz9DQGgCAggA
MB0GCWCGSAFlAwQBAgQQPCvupCCVI57DabGCXHCBbwSCAoCCHMMWy4IHvsdkABw2
<truncated>

You wrote:

However, if it didn't support GCM mode I would expect to get a compile time error telling me that EVP_aes_256_gcm() is not even defined, rather than a run-time error like this.

It is not that it does not support GCM mode in general, but it does not support GCM mode for encryption of the private key when doing PKCS#8. That limitation can not be known at compile time.

Upvotes: 5

Related Questions