CODEWITHSUNDEEP
copensslrsa
neubert
neubert

Reputation: 16792

getting a segmentation fault with RSA_public_encrypt

Here's my code:

#include <openssl/evp.h>
#include <openssl/pem.h>
#include <string.h>

int main (void)
{
        char publicKey[] = "-----BEGIN RSA PUBLIC KEY-----\n"\
"MIIBCgKCAQEAqBa0jeqfHO8CFZuHyN5WgdTd3uThkU/I9lR+bb2R9khVU1tYhiyL\n"\
"Gfnm051K0039gCBAz9S7HHO2lqR/B1kiEGwBzg83N3tL2UzQXXbpxJz0b7vV8rXZ\n"\
"fB9j+FCAtD5znittXRjWrNqlEyJyxK+PuxNF3uPGWSjkYtRKv2f5HCXsfRJah40w\n"\
"5zU+OKZsZMrSSxweYN9Jwc++oMaSPCsGKog6NPbkbcTK8Akveof8v5rpG50I/zcQ\n"\
"9dCD69qJcmhm4ai4fAZFJlsFH2HmxV6DERFs3TNyzGlSQ1gd/XLER7U9nlrOUT87\n"\
"mXreUIXSkAFdCrlHXHOzj0eodfN8IfHjnQIDAQAB\n"\
"-----END RSA PUBLIC KEY-----\n";
    char plaintext[] = "enp6";

    // base64 decode plaintext
    EVP_DecodeBlock(plaintext, plaintext, strlen(plaintext));

    BIO* bo = BIO_new(BIO_s_mem());
    BIO_write(bo, publicKey, strlen(publicKey));
    EVP_PKEY* pkey = 0;
    PEM_read_bio_PUBKEY(bo, NULL, NULL, NULL);
    BIO_free(bo);
    RSA* rsa = EVP_PKEY_get1_RSA(pkey);
    char ciphertext[RSA_size(rsa)];

    RSA_public_encrypt(strlen(plaintext), plaintext, ciphertext, rsa, RSA_NO_PADDING);
    printf("%d\n", (int) strlen(ciphertext));
}

I'm trying to run it by doing gcc -x c test2.c -lcrypto && ./a.out. Any ideas as to what I'm doing wrong?

Upvotes: 0

Views: 718

Answers (1)

Reinier Torenbeek
Reinier Torenbeek

Reputation: 17383

Your code contains several mistakes. The direct cause of the segfault is the fact that the variable pkey is never assigned any value after initializing it with 0. EVP_PKEY_get1_RSA(pkey) segfaults as a consequence. You probably intended to do

EVP_PKEY *pkey = PEM_read_bio_PUBKEY(bo, NULL, NULL, NULL);

However, with the PEM that you posted, that will not work. The header of the PEM indicates that this is an RSA key stored in "traditional" format. PEM_read_bio_RSAPublicKey is capable of reading such keys:

RSA *rsa = PEM_read_bio_RSAPublicKey(bo, NULL, NULL, NULL);

Then your choice of RSA_NO_PADDING when invoking RSA_public_encrypt is not right. In order to use that, the plaintext must be the exact same length as the key size. You could try using RSA_PKCS1_PADDING instead.

Finally, like mentioned in the comment section as well, using strlen on cipher text is incorrect, because it is not a 0-terminated string.

In general, you should check all return values for all OpenSSL functions invoked, to see when/if something went wrong.

Upvotes: 3

Related Questions