CODEWITHSUNDEEP
c++functionopenssl
mister
mister

Reputation: 3433

Return Function Crash - C++ Openssl

After returning the main function from this function, my program quits unexpectedly. After system("pause"), upon pressing enter, it will just exit from the program, it does not go back to main function. Hopefully someone can give some insight.

 bool DecryptKeyWithCheckSum(unsigned char *userKey,unsigned char *ivec,std::string cipherStr,unsigned char *genKey)
{

//split cipher
std::string first16Cipher = cipherStr.substr(0,AES_KEY_SIZE);
std::string last16Cipher = cipherStr.substr(AES_KEY_SIZE,cipherStr.length()-1);

//convert back to char
char cipherChar1[AES_KEY_SIZE];
strcpy(cipherChar1,first16Cipher.c_str());

//convert back to char
char cipherChar2[AES_KEY_SIZE];
strcpy(cipherChar2,last16Cipher.c_str());

//convert to unsigned char
unsigned char cipher1[AES_KEY_SIZE];
memcpy(cipher1,reinterpret_cast<unsigned char*>(cipherChar1),AES_KEY_SIZE);

unsigned char cipher2[AES_KEY_SIZE];
memcpy(cipher2,reinterpret_cast<unsigned char*>(cipherChar2),AES_KEY_SIZE);

//set key
AES_KEY key;
AES_set_encrypt_key(userKey, 128, &key);

unsigned char oriKey[AES_KEY_SIZE];
unsigned char checksum[AES_KEY_SIZE];

int num1 = 0;

//decrypt cipher
AES_cfb128_encrypt(cipher1, oriKey, AES_BLOCK_SIZE, &key, ivec, &num1,AES_DECRYPT);
AES_cfb128_encrypt(cipher2, checksum, AES_KEY_SIZE, &key, ivec, &num1,AES_DECRYPT);

//generate hash checksum
unsigned char hashChecksum[AES_KEY_SIZE];
this->hashData_MD5(oriKey,AES_KEY_SIZE,hashChecksum);

//convert checksum into string to compare
char checksum1[AES_KEY_SIZE];
strncpy(checksum1,reinterpret_cast<const char*>(hashChecksum),AES_KEY_SIZE);
checksum1[AES_KEY_SIZE] = '\0';
std::string checksum1Str = checksum1;

//convert checksum into string to compare
char checksum2[AES_KEY_SIZE];
strncpy(checksum2,reinterpret_cast<const char*>(checksum),AES_KEY_SIZE);
checksum2[AES_KEY_SIZE] = '\0';
std::string checksum2Str = checksum2;

//compare last 16 & checksum
if(checksum1Str==checksum2Str)
{
              //PROGRAM ABLE TO PRINT OUT THIS
    cout << "Decrypt Key Return true" << endl;
    system("pause");
    return true;
}
else
{
    cout << "Decrypt Key Return false" << endl;
    system("pause");
    return false;
}
}//end of function

int main()
{

//get user to enter password to decrypt password
unsigned char pass[AES_KEY_SIZE] = "password";
unsigned char tempKey[AES_KEY_SIZE];

//generate random key
unsigned char ckey[AES_KEY_SIZE];
this->generateRandomNum(ckey,AES_KEY_SIZE);
this->generateRandomNum(ivec,AES_IV_SIZE);

std::string cipherStr;
EncryptKeyWithCheckSum(pass,IV2,ckey,cipherStr);

if(DecryptKeyWithCheckSum(pass,IV,cipherStr,tempKey)==true)
{
          //BEFORE PRINTING OUT THIS, THE PROGRAM CRASHES UNEXPECTEDLY
    cout << "TRUE" << endl;
    system("pause");
}
else
{
    cout << "False" << endl;
    system("pause");
}
}

Upvotes: 1

Views: 283

Answers (1)

molbdnilo
molbdnilo

Reputation: 66371

Most of your arrays are one element too small.

For instance,

std::string first16Cipher = cipherStr.substr(0,AES_KEY_SIZE);

so first16Cipher has AES_KEY_SIZE characters

char cipherChar1[AES_KEY_SIZE];
strcpy(cipherChar1,first16Cipher.c_str());

But when you include the terminating zero, this line will copy AES_KEY_SIZE + 1 characters, which causes undefined behaviour, which makes your entire program undefined.

Upvotes: 1

Related Questions