CODEWITHSUNDEEP
copensslsegmentation-fault
James
James

Reputation: 187

Segmentation Fault While Encrypting with Openssl EVP: EVP_EncryptUpdate()

I want to use EVP and OpenSSL API to encrypt the binary data from a .exe file that I read into an unsigned char *. I'm not super familiar with this API and I'm afraid I'm doing something wrong that is causing the segmentation fault that I get when I compile. Here is my code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/conf.h>
#include <openssl/evp.h>
#include <openssl/err.h>
#include <openssl/aes.h>

void handleErrors(void){
  ERR_print_errors_fp(stderr);
  abort();
}

int main(){
  //read in the exe file and convert its bytes to a string
  FILE *inFile;
  inFile = fopen("Quasar.exe","rb");
  if (inFile == NULL){
    perror("Failed to read in file\n");
  }

  printf("File read in complete!\n");

  if(fseek(inFile , 0 , SEEK_END) == -1){
    perror("Offset error\n");
  };
    unsigned long lSize = ftell(inFile);
  if (lSize == -1){
    perror("Size error\n");
  }
    rewind(inFile);

  unsigned char *unencryptedText = (unsigned char*) malloc (sizeof(unsigned char)*lSize);
    fread(unencryptedText,1,lSize,inFile);
    fclose(inFile);

  unsigned char *encryptedText = (unsigned char*) malloc (3 *(sizeof(unsigned char)*lSize));

  //encrypt these bytes with open ssl
  printf("Encrypting...\n");
  int outlen, tmplen;
  unsigned char key[] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
  unsigned char iv[] = {1,2,3,4,5,6,7,8};
  EVP_CIPHER_CTX *ctx = NULL;

  EVP_CIPHER_CTX_init(ctx);
  EVP_EncryptInit_ex(ctx, EVP_idea_cbc(), NULL, key, iv);

  if(!EVP_EncryptUpdate(ctx, encryptedText, &outlen, unencryptedText, lSize)){
    return 0;
  }
      
  if(!EVP_EncryptFinal_ex(ctx, encryptedText + outlen, &tmplen)){
    return 0;
  }
  outlen += tmplen;
  EVP_CIPHER_CTX_cleanup(ctx);

  return 0;
}

Here is the console output:

File read in complete!
Encrypting...
zsh: segmentation fault  ./binary

Here is the Make script:

all: crypter.c 
    gcc -Wall -g -I/usr/local/include/openssl -L/usr/lib -lssl -lcrypto crypter.c -o binary
clean: 
    rm -f crypter

Upvotes: 0

Views: 1149

Answers (2)

dbush
dbush

Reputation: 223689

You're passing a NULL object to EVP_CIPHER_CTX_init This function expects the address of an existing EVP_CIPHER_CTX object.

You can fix this by creating an instance of EVP_CIPHER_CTX and passing its address to each function that needs it, or if you want to minimize changes to your code you can assign the address of this object to the existing ctx pointer.

EVP_CIPHER_CTX ctx_obj;
EVP_CIPHER_CTX *ctx = &ctx_obj;

EDIT:

Based on the comments, it seems you're using OpenSSL 1.1 or later. In that case, you want to instead use EVP_CIPHER_CTX_new to instantiate a new context:

EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();

Upvotes: 1

David Schwartz
David Schwartz

Reputation: 182733

EVP_CIPHER_CTX *ctx = NULL;

EVP_CIPHER_CTX_init(ctx);
EVP_EncryptInit_ex(ctx, EVP_idea_cbc(), NULL, key, iv);

At no time do you assign ctx any value other than NULL. You pass a NULL to both of the functions above.

What you probably wanted was:

ECP_CIPHER_CTX ctx;
EVP_CIPHER_CTX_init(&ctx);

Notice how this doesn't pass NULL to EVP_CIPHER_CTX_init?

Upvotes: 1

Related Questions