Understanding engine initialization in OpenSSL

Question

I'm trying to set up a basic test of HMAC-SHA-256 hashing but I'm having problems with the engine setup. Ideally I would like to set up only the HMAC-SHA-algorithm but so far I haven't even gotten the general case where load all the algorithms to work. Currently I'm getting segfaults on the row where I try to set the default digests.

Also, I'm regularly a Java guy, so don't hesitate to point out any mistakes in the code.

#include <openssl/hmac.h>
#include <openssl/evp.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main() {
  unsigned char* key = (unsigned char*) "0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b";
  unsigned char* data = (unsigned char*) "4869205468657265";
  unsigned char* expected = (unsigned char*) "b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7";
  unsigned char* result;
  HMAC_CTX* ctx;
  ENGINE* e;

  ENGINE_load_builtin_engines();
  ENGINE_register_all_complete();
  ENGINE_set_default_digests(e);

  HMAC_CTX_init(ctx);
  HMAC_Init_ex(ctx, key, 40, EVP_sha256(), e);
  result = HMAC(NULL, NULL, 40, data, 16, NULL, NULL);
  HMAC_CTX_cleanup(ctx);

  ENGINE_finish(e);
  ENGINE_free(e);

  if (strcmp((char*) result, (char*) expected) == 0) {
    printf("Test ok\n");
  } else {
    printf("Got %s instead of %s\n", result, expected);
  }
}

EDIT: The program has now evolved to the following, but I'm still segfaulting at HMAC_Init_ex:

unsigned char* key = (unsigned char*) "0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b";
unsigned char* data = (unsigned char*) "4869205468657265";
unsigned char* expected = (unsigned char*) "b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7";
unsigned char* result;
unsigned int result_len = 64;
HMAC_CTX ctx;
ENGINE* e;

result = (unsigned char*) malloc(sizeof(char) * result_len);
e = (ENGINE*) ENGINE_new();

ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
ENGINE_set_default_digests(e);

HMAC_CTX_init(&ctx);
HMAC_Init_ex(&ctx, key, 16, EVP_sha256(), e); 
HMAC_Update(&ctx, data, 40);
HMAC_Final(&ctx, result, &result_len);
HMAC_CTX_cleanup(&ctx);

ENGINE_finish(e);
ENGINE_free(e);

Answer 1

The problem with your original suggestion is, as Martin said, that you need to initialise the ENGINE. The problem with your edited code was that you were doing ENGINE_new, which is getting you a completely new ENGINE of your own, which you then need to provide with cipher methods, digest methods, etc. In fact, for what you want (and what almost everybody wants), just completely ignoring all of the ENGINE stuff is the right choice.

Some subsidiary problems:

• your strings were hex, but you needed a \x per character to actually get that hex byte at that position in the string, which I suspect was what you wanted.
• you were trying to hash 40 bytes from "data", which wasn't that long (actual effect: you'd end up partly hashing your result string)
• your expected result was (as far as I can tell) incorrect
• you would print out random characters to the terminal, since the HMAC function will produce 32 bytes of random binary data, not printable stuff.

The following code compiles, works and passes the test. It's a bit different to the example code you found (since it still uses the individual HMAC_* functions - useful if you want to do your hashing bit by bit using HMAC_Update):

#include <openssl/engine.h>
#include <openssl/hmac.h>
#include <openssl/evp.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>


int main(void)
{
        unsigned char* key = (unsigned char*) "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b";
        unsigned char* data = (unsigned char*) "\x48\x69\x20\x54\x68\x65\x72\x65";
        unsigned char* expected = (unsigned char*) "\x49\x2c\xe0\x20\xfe\x25\x34\xa5\x78\x9d\xc3\x84\x88\x06\xc7\x8f\x4f\x67\x11\x39\x7f\x08\xe7\xe7\xa1\x2c\xa5\xa4\x48\x3c\x8a\xa6";
        unsigned char* result;
        unsigned int result_len = 32;
        int i;
        HMAC_CTX ctx;

        result = (unsigned char*) malloc(sizeof(char) * result_len);

        ENGINE_load_builtin_engines();
        ENGINE_register_all_complete();

        HMAC_CTX_init(&ctx);
        HMAC_Init_ex(&ctx, key, 16, EVP_sha256(), NULL);
        HMAC_Update(&ctx, data, 8);
        HMAC_Final(&ctx, result, &result_len);
        HMAC_CTX_cleanup(&ctx);

        for (i=0; i!=result_len; i++)
        {
                if (expected[i]!=result[i])
                {
                        printf("Got %02X instead of %02X at byte %d!\n", result[i], expected[i], i);
                        break;
                }
        }
        if (i==result_len)
        {
                printf("Test ok!\n");
        }
        return 0;
}

Of course, it doesn't answer your original question about how to initialise ENGINEs, but there's really no right answer to that without having more context, which context turns out not to be relevant in your situation...

Answer 2

Okay, turns out that you don't have to use an engine but I'd misunderstood exactly how not to use an explicit engine. I also misunderstood how to properly format the test vectors. In the end I looked at hmactest.c which pretty much does all I want to do, I just didn't understand the code.

The final solution to what I was trying to do looks like this:

int main() {
  unsigned char* key = (unsigned char*) "Jefe";
  unsigned char* data = (unsigned char*) "what do ya want for nothing?";
  unsigned char* expected = (unsigned char*) "5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843";
  unsigned char* result;
  unsigned int result_len = 32;
  int i;
  static char res_hexstring[32];

  result = HMAC(EVP_sha256(), key, 4, data, 28, NULL, NULL);
  for (i = 0; i < result_len; i++) {
    sprintf(&(res_hexstring[i * 2]), "%02x", result[i]);
  }

  if (strcmp((char*) res_hexstring, (char*) expected) == 0) {
    printf("Test ok, result length %d\n", result_len);
  } else {
    printf("Got %s instead of %s\n", res_hexstring, expected);
  }
}

But since I was asking about something entirely different, I'm unsure about what to do with the original question. Suggestions?

Answer 3

It looks as though nothing is allocating an engine, so the first use of e is segfaulting. I think you need to call ENGINE *ENGINE_new(void) first.

(Note that I've used OpenSSL, but I haven't used the ENGINE functions before.)

Update: I'm not really happy with my own answer (I had to dash off to tea, before). So my further notes are:

1. I've had a bit of a look at the (long) man page for the ENGINE functions, and I'm not quite sure that calling ENGINE_new is sufficient.

2. I didn't notice that the calls to the HMAC_CTX_* functions were taking an uninitialized pointer, rather than a pointer to an allocated structure. HMAC_CTX_init will try to write to the memory pointed to by its ctx parameter, which will segfault. You need to declare and use ctx like this:

   HMAC_CTX ctx;
   HMAC_CTX_init(&ctx);
   HMAC_Init_ex(&ctx, key, 40, EVP_sha256(), e);
   ...
   

   That way you're allocating the structure on the stack, and then passing a pointer to it.

3. The HMAC function doesn't take a pointer to a CTX at all, so apart from global or thread-local storage, I'm not sure what it's connection to the CTX is. I think you can bypass that by calling HMAC_Update one-or-more times, followed by HMAC_Final to get the result. You'd need to allocate space for that result, so something like the following would work for that:

   unsigned int len;
   HMAC_Final(&ctx, result, &len);