CODEWITHSUNDEEP
cencryptionopensslrsa
Pudron
Pudron

Reputation: 96

How to use OpenSSL 3.0 RSA in C?

I want to simply encrypt and decrypt some data. Many old methods have been deprecated since OpenSSL 3.0.
The public and private keys generation code:

void generateKeys(){
    EVP_PKEY*pkey=EVP_RSA_gen(1024);
    if(pkey==NULL){
        fprintf(stderr,"error: rsa gen\n");
        ERR_print_errors_fp(stderr);
        return;
    }
    FILE*fp=fopen("public.txt","wt");
    if(fp!=NULL){
        PEM_write_PUBKEY(fp,pkey);
        fclose(fp);
    }else{
        perror("file error");
    }
    fp=fopen("private.txt","wt");
    if(fp!=NULL){
        PEM_write_PrivateKey(fp,pkey,NULL,NULL,0,NULL,NULL);
        fclose(fp);
    }else{
        perror("file error");
    }
    EVP_PKEY_free(pkey);
}

The encrypt function:

uchar*encrypt(uchar*src,uint len,int*length){
    FILE*fp=fopen("public.txt","r");
    if(fp==NULL){
        perror("file error");
        return NULL;
    }
    EVP_PKEY*pkey;
    pkey=PEM_read_PUBKEY(fp,NULL,NULL,NULL);
    fclose(fp);
    if(pkey==NULL){
        fprintf(stderr,"error: read publics key\n");
        return NULL;
    }
    EVP_PKEY_CTX*ctx=EVP_PKEY_CTX_new(pkey,NULL);
    EVP_PKEY_encrypt_init(ctx);
    uchar*dst=(uchar*)malloc(2048);
    size_t outl;
    if(!EVP_PKEY_encrypt(ctx,dst,&outl,src,(size_t)len)){
        fprintf(stderr,"error: encrypt\n");
        EVP_PKEY_free(pkey);
        free(dst);
        return NULL;
    }
    int len2=outl;
    EVP_PKEY_free(pkey);
    EVP_PKEY_CTX_free(ctx);
    BIO_dump_fp(stdout,dst,len2);
    printf("len: %d, len2: %d\n",len,len2);
    if(length!=NULL){
        *length=len2;
    }
    return dst;
}

The decrypt function:

uchar*decrypt(uchar*src,int len){
    FILE*fp=fopen("private.txt","r");
    if(fp==NULL){
        perror("file error");
        return NULL;
    }
    EVP_PKEY*pkey=PEM_read_PrivateKey(fp,NULL,NULL,NULL);
    fclose(fp);
    if(pkey==NULL){
        fprintf(stderr,"error: read private key\n");
        return NULL;
    }
    EVP_PKEY_CTX*ctx=EVP_PKEY_CTX_new(pkey,NULL);
    EVP_PKEY_decrypt_init(ctx);
    uchar*dst=(uchar*)malloc(2048);
    size_t outl;
    size_t inl=len;
    if(!EVP_PKEY_decrypt(ctx,dst,&outl,src,inl)){
        fprintf(stderr,"error: decrypt\n");
        free(dst);
        dst=NULL;
    }else{
        BIO_dump_fp(stdout,dst,(int)outl);
        printf("len: %d, outl: %lld\n",len,outl);
    } 
    EVP_PKEY_free(pkey);
    EVP_PKEY_CTX_free(ctx);
    return dst;
}

The function EVP_PKEY_decrypt always returns 0. What's the problem?
The official document suggest the EVP module. So I try to use this module to crypt some data. The default RSA padding is RSA_PKCS1_PADDING. But it seems that there are no complete tutorial about how to use RSA by EVP.

Upvotes: 2

Views: 6197

Answers (2)

user3059552
user3059552

Reputation: 71

EVP_PKEY_decrypt presents 2 related invocation idioms:

  1. call with a NULL output buffer (dst in your example) and a pointer to a size_t arg (&outl) will populate the size_t arg with the expcted decrypted size, to let you know how much memory to allocate.
  2. call with the allocated output buffer and a pointer to the needed memory size argument having been set then decrypts into the out buffer.

your code skipped the first call, since you knew the decrypted size already. so when you initialized outl = 2048 you essentially accomplished what the full 2-step process would have done, and that's why it suddenly started working.

the page: https://www.openssl.org/docs/manmaster/man3/EVP_PKEY_decrypt.html shows an example (near the bottom).

Upvotes: 2

Pudron
Pudron

Reputation: 96

It is amazing that it works well when I change the code size_t outl to size_t outl=2048 in the decrypt function.

Upvotes: 1

Related Questions