Memory leak in Crypto++ RSAES class

Question

I'm trying to learn how to use Crypto++ classes. My goal is to generate public and private keys for RSA encryption and then do basic encryption and decryption of a plaintext.

So I'm taking their example from here -- "RSA Encryption Scheme (OAEP and SHA) using Filters", slightly modified for readability:

This parts works OK:

CryptoPP::AutoSeededRandomPool rng;

//Generate Parameters
CryptoPP::InvertibleRSAFunction params;
params.GenerateRandomWithKeySize(rng, 3072);

//Create Keys
CryptoPP::RSA::PrivateKey privateKey(params);
CryptoPP::RSA::PublicKey publicKey(params);

std::string plain="Hello world!", cipher, recovered;

//Encryption
CryptoPP::RSAES_OAEP_SHA_Encryptor e(publicKey);

But then when I call this block:

CryptoPP::StringSink* pSS = new CryptoPP::StringSink( cipher );
CryptoPP::PK_EncryptorFilter* pEF = new CryptoPP::PK_EncryptorFilter( rng, e, pSS);

CryptoPP::StringSource ss1( plain, true, pEF);

It causes the memory leak(s). I get the following in the Visual Studio output window:

Detected memory leaks!
Dumping objects ->
{24781} normal block at 0x029BCFF8, 28 bytes long.
 Data: <class CryptoPP::> 63 6C 61 73 73 20 43 72 79 70 74 6F 50 50 3A 3A 
{24780} normal block at 0x029BCFB0, 8 bytes long.
 Data: <        > F8 CF 9B 02 00 00 00 00 
Object dump complete.

OK, so I did the most obvious thing and added these:

delete pEF;
delete pSS;

but it caused an unhandled exception, so I assumed that one of the destructors in Crypto++ classes took care of deleting some of those objects.

So the question is -- where is this leak coming from?

I tried stepping into StringSink, PK_EncryptorFilter and StringSource with a Visual Studio debugger to see what's going on, but the code is quite convoluted to figure it out right away.

Any idea how to fix those memory leaks?

Answer 1

It causes the memory leak(s). I get the following in the Visual Studio output window:

Detected memory leaks!
Dumping objects ->
{24781} normal block at 0x029BCFF8, 28 bytes long.
 Data: <class CryptoPP::> 63 6C 61 73 73 20 43 72 79 70 74 6F 50 50 3A 3A 
{24780} normal block at 0x029BCFB0, 8 bytes long.
 Data: <        > F8 CF 9B 02 00 00 00 00 
Object dump complete.

The code you used looks a tad bit unusual, but I believe its well formed.

I believe what you are seeing is Microsoft's decades old bug typeinfo.name() memory leaks. Its been around since the VC 5.0 or VC 6.0 days.

---

CryptoPP::StringSink* pSS = new CryptoPP::StringSink( cipher );
CryptoPP::PK_EncryptorFilter* pEF = new CryptoPP::PK_EncryptorFilter( rng, e, pSS);

CryptoPP::StringSource ss1( plain, true, pEF);

Here's what a pipeline often looks like:

CryptoPP::StringSource ss( plain, true,
    new CryptoPP::PK_EncryptorFilter( rng, e,
        new CryptoPP::StringSink( cipher )));

Everything that follows the code above is a red herring. You went down a rabbit hole because Microsoft won't fix their bugs.

---

OK, so I did the most obvious thing and added these:

delete pEF;
delete pSS;

but it caused an unhandled exception

Yeah, that was not right. From the Readme.txt:

* Important Usage Notes *

1. If a constructor for A takes a pointer to an object B (except primitive types such as int and char), then A owns B and will delete B at A's destruction. If a constructor for A takes a reference to an object B, then the caller retains ownership of B and should not destroy it until A no longer needs it.

2. Crypto++ is thread safe at the class level. This means you can use Crypto++ safely in a multithreaded application, but you must provide synchronization when multiple threads access a common Crypto++ object.

pEF and pSS were pointers, and they were owned by someone else. They got deleted twice, which caused the exception.

---

Memory leak in Crypto++ RSAES class ...

If you run the cryptest.exe program, then you will see 60 or 80 leaks reported. I've tried to find a solution to that bug for about 10 or 15 years. Most recently was How to remediate Microsoft typeinfo.name() memory leaks? on Stack Overflow.

EDIT also see Windows Debug build memory leaks cleared on the user list and Commit 301437e693fe8bff. The library moved to static initialization from dynamic initialization to avoid problems on Microsoft platforms. The static initializer list is accessed with inti_seg on Windows; and constructor and init_priority attributes with GCC.

Its a "best effort" to use static initialization, if available. Otherwise, things fall back to dynamic initialization. Here, "static initialization" means getting the library into the CRT static initialization list that runs constructor functions and calls global object ctors (and not a vanilla C++ static object).