CODEWITHSUNDEEP
openssl
Nathan Osman
Nathan Osman

Reputation: 73195

How to abort RSA_generate_key?

Consider the following invocation of RSA_generate_key():

RSA * rsa = RSA_generate_key(8192, RSA_F4, NULL, NULL);

Generating an 8,192-bit RSA key can take a long time (anywhere from a few seconds to a few minutes). Suppose the application containing the line of code above offers the user a button that will cancel the key generation.

How can I abort the computation and make the function return before the key has been generated? I remember the third argument to RSA_generate_key() is a callback function used for displaying progress - is there any way that callback could return a value that means "abort the operation and return"?

Running the function in another thread and then terminating the thread is not an option.

Upvotes: 8

Views: 3182

Answers (6)

user4815162342
user4815162342

Reputation: 155046

Since RSA_generate_key provides a progress callback, you can longjmp out of it to terminate the function. With a bit of additional code, you can create a wrapper for RSA_generate_key that accepts a generic test function which can be used to check for a timeout or a flag set by the windowing system.

#include <openssl/rsa.h>
#include <stdbool.h>
#include <setjmp.h>

struct trampoline_ctx {
  bool (*testfn)(void *);
  void *testfn_arg;
  jmp_buf env;
};

static void trampoline(int ignore1, int ignore2, void *arg)
{
  struct trampoline_ctx *ctx = arg;
  if (!ctx->testfn(ctx->testfn_arg))
    longjmp(ctx->env, 1);
}

// like RSA_generate_key, but accepts a test function. If testfn returns
// false, key generation is terminated and NULL is returned.    
RSA *
my_generate_key(int num, unsigned long e,
                bool (*testfn)(void *), void *testfn_arg)
{
  struct trampoline_ctx ctx;
  ctx.testfn = testfn;
  ctx.testfn_arg = testfn_arg;
  if (setjmp(ctx.env))
    return NULL;
  return RSA_generate_key(num, e, trampoline, &ctx);
}

This approach is surprisingly portable, as longjmp is mandated by both C89 and C99. Its disadvantage is that it can leak resources if the function you're longjmping out of allocates them dynamically. However, in practice the leak can be small enough to be unnoticable if done infrequently or only on explicit user request. To be positive that this is the case, run the code in a tight loop and observe resource consumption of the process.

Here is a test program for the above function:

#include <stdio.h>
#include <sys/time.h>

double now()
{
  struct timeval tv;
  gettimeofday(&tv, NULL);
  return tv.tv_sec + (double) tv.tv_usec / 1e6;
}

struct tt_ctx {
  double start;
  double limit;
};

bool test_time_limit(void *arg)
{
  struct tt_ctx *ctx = arg;
  return now() - ctx->start <= ctx->limit;
}

int main(int argc, char **argv)
{
  int limit = atoi(argv[1]);
  struct tt_ctx ctx;
  ctx.start = now();
  ctx.limit = limit / 1000.0;

  RSA *key = my_generate_key(4096, 65537, test_time_limit, &ctx);
  printf("%p\n", key);
  return 0;
}

The test program assumes POSIX gettimeofday, but can be trivially converted to another high-resolution clock as provided by the system. Test procedure is as follows:

Append both pieces of code to a file and compile with -lrsa. The test program will generate a 4096-bit RSA key within the time limit specified in milliseconds on the command line. In all cases it prints the resulting RSA * pointer to indicate whether my_generate_key completed its request or it was aborted. The output of time accompanies execution as a sanity check to verify that the time limit is being respected:

# try with a 10ms limit
$ time ./a.out 10
(nil)                         # too short
./a.out 10  0.02s user 0.00s system 85% cpu 0.023 total

# see if 100ms is enough time
$ time ./a.out 100
(nil)                         # still too short
./a.out 100  0.10s user 0.00s system 97% cpu 0.106 total

# try with 1 whole second:
$ time ./a.out 1000
0x2369010                     # success!
./a.out 1000  0.64s user 0.00s system 99% cpu 0.649 total

Upvotes: 7

Troy Alford
Troy Alford

Reputation: 27236

I believe the problem is that a callback doesn't generally give you the ability to send information in, only receive information back. As some of the other posters have mentioned, you might consider altering the source code of the actual key generation routine, and compiling your own version. In this case, you could pass some kind of a value by reference back to your calling routine, which you could then set outside, and thus trigger a fail condition.

Note: RSA_generate_key is actually deprecated, and is now simply a wrapper for RSA_generate_key_ex.

According to version 1.19.4.2 of the rsa_gen.c file, unless you are in FIPS_mode, the key is going to be generated by the static method rsa_builtin_keygen. In FIPS_mode, it will be generated by whatever rsa->meth->rsa_keygen is upon entering the RSA_generate_key.

That said, there are a number of places where the callback (cb parameter) is passed out to other methods, presumably so that those methods can, in turn, call it to update status.

The trick would be updating either the BN_GENCB to contain some kind of a "cancel" flag, or alter the way in which the callback is actually called within the method, so that when the callback method fires, you can set a flag, which the calling function would honor by breaking out of the generation subroutine.

How you'd go about that is still something you need to figure out and work through - but hopefully this will give you a bit of a start.

Upvotes: 2

flup
flup

Reputation: 27104

If you do not wish to terminate the thread, you can fork off a new process instead and terminate that if the user aborts?

Upvotes: 0

whunmr
whunmr

Reputation: 2435

According source code, without modify openssl's code, we can not interrupt the RSA_generate_key method to let it return.

I thought two ways to do this:

  1. let the method running in background thread, and when user presses the cancel button, let the method run to end in background. and let the thread die afterwards. and ignore the results. and start a new thread if user request another generation.

  2. Another one is to start a sub-process to generate the key. the RSA structure contains lots of BIGNUM member, which is also structure of some ints/ulongs, after sub-process generated the keys, copy these relevant members back to parent-process. if user presses the cancel button, then kill the sub-process is safe. But not sure whether copying these ints/longs is enough for your senario.

Upvotes: 0

doptimusprime
doptimusprime

Reputation: 9405

Although I never face such situation and never did anything before. You can try the following if you can modify the source code of OpenSSL:

Let us say a variable V is shared between two functions. So either it is a global variable or static variable in the file and being used by RSA_generate_key and say another function RSA_set_V.

RSA_generate_key will only read the variable. RSA_set_V sets the variable.

In RSA_generate_key, the time consuming loop may use an additional condition of checking this variable. It may terminate this loop or exit the function on a certain value of V. e.g.

  while (V!=CERTAIN_VALUE && Rest_of_Condition) {
      //Loop body.
  }

Now, in RSA_set_V, set V to CERTAIN_VALUE. When user clicks on Stop, call RSA_set_V. It should stop RSA_generate_key.

However, it has few problems. It may slow down RSA_generate_key.

Upvotes: 1

charliehorse55
charliehorse55

Reputation: 1990

The progress callback can't be used to cancel the function, it's purely for displaying the progress to the user. (Read more here)

Spawn a thread to call RSA_generate_key() so that it executes in the background. When the user presses the cancel button, kill the thread.

Upvotes: 2

Related Questions