Crypto++ extra block at the end of encrypted stream in AES-128 CBC mode

Question

I'm trying to encrypt 320 bytes of binary data using AES-128 in CBC mode and store the cipher into a file. The output file should have been of 320 bytes, but I got 336 bytes. Here is my code:

#include <iostream>
#include <fstream>
#include <crypto++/aes.h>
#include <crypto++/modes.h>
#include <crypto++/base64.h>
#include <crypto++/sha.h>
#include <cryptopp/osrng.h>
#include <crypto++/filters.h>
#include <crypto++/files.h>

namespace CryptoPP
{
    using byte = unsigned char;
}

void myAESTest()
{
    std::string password = "testPassWord";

    // hash the password string
    // -------------------------------
    CryptoPP::byte key[CryptoPP::AES::DEFAULT_KEYLENGTH], iv[CryptoPP::AES::BLOCKSIZE];
    CryptoPP::byte passHash[CryptoPP::SHA256::DIGESTSIZE];
    CryptoPP::SHA256().CalculateDigest(passHash, (CryptoPP::byte*) password.data(), password.size());
    std::memcpy(key, passHash, CryptoPP::AES::DEFAULT_KEYLENGTH);
    std::memcpy(iv, passHash+CryptoPP::AES::DEFAULT_KEYLENGTH, CryptoPP::AES::BLOCKSIZE);

    // encrypt
    // ---------------------------------
    int chunkSize = 20*CryptoPP::AES::BLOCKSIZE;

    CryptoPP::CBC_Mode<CryptoPP::AES>::Encryption encryptor;
    encryptor.SetKeyWithIV(key, sizeof(key), iv);

    std::ofstream testOut("./test.enc", std::ios::binary);
    CryptoPP::FileSink outSink(testOut);

    CryptoPP::byte message[chunkSize];

    CryptoPP::StreamTransformationFilter stfenc(encryptor, new CryptoPP::Redirector(outSink));

    for(int i = 0; i < chunkSize; i ++)
    {
        message[i] = (CryptoPP::byte)i;
    }

    stfenc.Put(message, chunkSize);
    stfenc.MessageEnd();
    testOut.close();

    // decrypt
    // ------------------------------------
    // Because of some unknown reason increase chuksize by 1 block
    // chunkSize+=16;

    CryptoPP::byte cipher[chunkSize], decrypted[chunkSize];

    CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryptor;
    decryptor.SetKeyWithIV(key, sizeof(key), iv);

    std::ifstream inFile("./test.enc", std::ios::binary);
    inFile.read((char *)cipher, chunkSize);

    CryptoPP::ArraySink decSink(decrypted, chunkSize);
    CryptoPP::StreamTransformationFilter stfdec(decryptor, new CryptoPP::Redirector(decSink));

    stfdec.Put(cipher, chunkSize);
    stfdec.MessageEnd();
    inFile.close();

    for(int i = 0; i < chunkSize; i++)
    {
        std::cout << (int)decrypted[i] << ' ';
    }
    std::cout << std::endl;
}

int main(int argc, char* argv[]) 
{
    myAESTest();
    return 0;
}

I'm not able to understand how the last 16 bytes are generated. If I choose to ignore the last 16 bytes in the decryption, CryptoPP throws CryptoPP::InvalidCiphertext error:

terminate called after throwing an instance of 'CryptoPP::InvalidCiphertext'
  what():  StreamTransformationFilter: invalid PKCS #7 block padding found

Answer 1

I'm not able to understand how the last 16 bytes are generated. If I choose to ignore the last 16 bytes in the decryption, Crypto++ throws InvalidCiphertext error

The last 16 bytes are padding. Padding is added by the StreamTransformationFilter filter; see StreamTransformationFilter Class Reference in the manual. Though not obvious, DEFAULT_PADDING is PKCS_PADDING for ECB_Mode and CBC_Mode. It is NO_PADDING for other modes like OFB_Mode and CTR_Mode.

You only need to specify NO_PADDING for both the encryption and decryption filters. However, you have to ensure the plaintext and ciphertext are a multiple of the blocksize, which is 16 for AES.

You can sidestep the blocksize restriction by switching to another mode like CTR_Mode. But then you have to be very careful of key or IV reuse, which may be difficult due to the password derivation scheme you are using.

So instead of:

CBC_Mode<AES>::Encryption encryptor;
...
StreamTransformationFilter stfenc(encryptor, new Redirector(outSink));

Use:

CBC_Mode<AES>::Encryption encryptor;
...
StreamTransformationFilter stfenc(encryptor, new Redirector(outSink), NO_PADDING);

Also see CBC_Mode on the Crypto++ wiki. You might also be interested in Authenticated Encryption on the wiki.

---

For this, you can also:

#ifndef CRYPTOPP_NO_GLOBAL_BYTE
namespace CryptoPP
{
    using byte = unsigned char;
}
#endif

CRYPTOPP_NO_GLOBAL_BYTE is defined after the C++17 std::byte fixes. If CRYPTOPP_NO_GLOBAL_BYTE is not defined, then byte is in the global namespace (Crypto++ 5.6.5 and earlier). If CRYPTOPP_NO_GLOBAL_BYTE is defined, then byte is in the CryptoPP namespace (Crypto++ 6.0 and later).

---

For this:

std::ofstream testOut("./test.enc", std::ios::binary);
FileSink outSink(testOut);

You can also do:

FileSink outSink("./test.enc");

---

For this:

SHA256().CalculateDigest(passHash, (byte*) password.data(), password.size());
std::memcpy(key, passHash, AES::DEFAULT_KEYLENGTH);
std::memcpy(iv, passHash+AES::DEFAULT_KEYLENGTH, AES::BLOCKSIZE);

You might consider using HKDF as a derivation function. Use the one password but two different labels to ensure independent derivation. One label might be the string "AES key derivation version 1" and the other label might be "AES iv derivation version 1".

The label would be used as the info parameter for DeriveKey. You just need to call it twice, once for the key and once for the iv.

unsigned int DeriveKey (byte *derived, size_t derivedLen,
    const byte *secret, size_t secretLen,
    const byte *salt, size_t saltLen,
    const byte *info, size_t infoLen) const

secret is the password. If you have salt then use it. Otherwise HKDF uses a default salt.

Also see HKDF on the Crypto++ wiki.

---

Finally, regarding this:

You can sidestep the blocksize restriction by switching to another mode like CTR_Mode. But then you have to be very careful of key or IV reuse, which may be difficult due to the password derivation scheme you are using.

You might also consider an Integrated Encryption Scheme, like Elliptic Curve Integrated Encryption Scheme. It is IND-CCA2, which is a strong notion of security. Everything you need is packaged into the encryption scheme.

Under ECIES each user gets a public/private keypair. Then, a large random secret is used as a seed for the AES key, iv and mac key. The the plaintext is encrypted and authenticated. Finally, the seed is encrypted under the user's public key. The password is still used, but it is used to decrypt the private key.