CODEWITHSUNDEEP
c++opensslrsa
void.pointer
void.pointer

Reputation: 26315

EVP_PKEY_encrypt() is giving output size of 0

I have a small program that I am writing to generate a 1024-bit RSA key and encrypt some data. Per the EVP_PKEY_encrypt() documentation, I invoke it once with a null out parameter to get the size of the output buffer for the encryption. However, in my case, it's giving me 0. If I write the public key out as a PEM file, create a new context, and load that key into the context via BIO_read(), then I get an output size of 128 which still seems wrong since my test data is a very simple string: "SecretMessage".

What am I doing wrong here? Why am I getting 0 back for the outlen parameter?

   int m_keyBits = 1024;
   int m_padding = RSA_PKCS1_OAEP_PADDING;
   EVP_PKEY* m_key{};
   auto m_context = EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, nullptr);

   if (EVP_PKEY_keygen_init(m_context) <= 0)
   {
      LogFailure();
      return;
   }

   if (EVP_PKEY_CTX_set_rsa_keygen_bits(m_context, m_keyBits) <= 0)
   {
      LogFailure();
      return;
   }

   if (EVP_PKEY_keygen(m_context, &m_key) <= 0)
   {
      LogFailure();
      return;
   }

   std::string const originalData = "SecretMessage";

   std::vector<std::uint8_t> dst;
   std::string const& src = originalData;

   if (EVP_PKEY_encrypt_init(m_context) <= 0)
   {
      LogFailure();
      return;
   }

   if (EVP_PKEY_CTX_set_rsa_padding(m_context, m_padding) <= 0)
   {
      LogFailure();
      return;
   }

   // Invoke encrypt method with NULL output buffer pointer which means OpenSSL will tell us the
   // maximum buffer size.
   std::size_t maxSize;
   if (EVP_PKEY_encrypt(m_context, nullptr, &maxSize,
      reinterpret_cast<unsigned char const*>(&src[0]), src.size()) <= 0)
   {
      LogFailure();
      return;
   }

   dst.resize(maxSize);
   std::size_t writtenSize = maxSize;

   if (EVP_PKEY_encrypt(m_context, reinterpret_cast<unsigned char*>(&dst[0]), &writtenSize,
      reinterpret_cast<unsigned char const*>(&src[0]), src.size()) <= 0)
   {
      LogFailure();
      return;
   }

   dst.resize(writtenSize);

Upvotes: 1

Views: 1479

Answers (1)

void.pointer
void.pointer

Reputation: 26315

Apparently you can't reuse the context for multiple operations. You have to create a context for the keygen, then free it, then create a new one for the encryption, then free it. The OpenSSL unit tests (specifically enginetest.c) do it this way, and that's how I found this out.

Upvotes: 1

Related Questions