AES+HMAC encryption in multiple threads - Java

Question

I'm developing a little program to encryp/decrypt a binary file using AES-256 and HMAC to check the results.

My code is based on AESCrypt implementation in Java, but I wanted to modify it to allow multiple threads to do the job simultaneously.

I get the size of original bytes and calculate the number of 16 bytes blocks per thread, then I startes the threads with information about the offset to apply for reading and writing (because there is a header for the encrypted file, so the offset_write = offset_read+header_length).

When it finishes the encryption I passed the output content (without the header) trough the HMAC to generate the checksum.

The problem is that some bytes get corrupted in the bytes between two threads.

Code of main:

//..
// Initialization and creation of iv, aesKey
//..

in = new FileInputStream(fromPath);
out = new FileOutputStream(toPath);

//..
// Some code for generate the header and write it to out
//..
double totalBytes = new Long(archivo.length()).doubleValue();
int bloquesHilo = new Double(Math.ceil(totalBytes/(AESCrypt.NUM_THREADS*AESCrypt.BLOCK_SIZE))).intValue();
int offset_write = new Long((out.getChannel()).position()).intValue();

for (int i = 0; i < AESCrypt.NUM_THREADS; i++)
{
    int offset = bloquesHilo*AESCrypt.BLOCK_SIZE*i;
    HiloCrypt hilo = new HiloCrypt(fromPath, toPath, ivSpec, aesKey, offset, offsetInicio, bloquesHilo, this);
    hilo.start();
}

Code for a thread (class HiloCrypt): public class HiloCrypt extends Thread {

    private RandomAccessFile in;
    private RandomAccessFile out;

    private Cipher cipher;
    private Mac hmac;
    private IvParameterSpec ivSpec2;
    private SecretKeySpec aesKey2;

    private Integer num_blocks;
    private Integer offset_read;
    private Integer offset_write;

    private AESCrypt parent;

    public HiloCrypt(String input, String output, IvParameterSpec ivSpec, SecretKeySpec aesKey, Integer offset_thread, Integer offset_write, Integer blocks, AESCrypt parent2) 
    {
        try
        {
                        // If i don't use RandomAccessFile there is a problem copying data
            this.in = new RandomAccessFile(input, "r");
            this.out = new RandomAccessFile(output, "rw");

            int total_offset_write = offset_write + offset_thread;

                        // Adjust the offset for reading and writing 
            this.out.seek(total_offset_write);
            this.in.seek(offset_thread);

            this.ivSpec2 = ivSpec;
            this.aesKey2 = aesKey;

            this.cipher = Cipher.getInstance(AESCrypt.CRYPT_TRANS);
            this.hmac = Mac.getInstance(AESCrypt.HMAC_ALG);

            this.num_blocks = blocks;
            this.offset_read = offset_thread;
            this.offset_write = total_offset_write;
            this.parent = parent2;

        } catch (Exception e)
        {
            System.err.println(e);
            return;
        }
    }


    public void run()
        {
        int len, last,block_counter,total = 0;
        byte[] text = new byte[AESCrypt.BLOCK_SIZE];

        try{
            // Start encryption objects
            this.cipher.init(Cipher.ENCRYPT_MODE, this.aesKey2, this.ivSpec2);
            this.hmac.init(new SecretKeySpec(this.aesKey2.getEncoded(), AESCrypt.HMAC_ALG));

            while ((len = this.in.read(text)) > 0 && block_counter < this.num_blocks) 
            {
                this.cipher.update(text, 0, AESCrypt.BLOCK_SIZE, text);
                this.hmac.update(text);

                // Write the block
                this.out.write(text);

                last = len;
                total+=len;

                block_counter++;
            }

            if (len < 0) // If it's the last block, calculate the HMAC
            {
                last &= 0x0f;
                this.out.write(last);

                this.out.seek(this.offset_write-this.offset_read);

                while ((len = this.out.read(text)) > 0) 
                {
                    this.hmac.update(text);
                }

                // write last block of HMAC
                text=this.hmac.doFinal();
                this.out.write(text);
            }

                        // Close streams
            this.in.close();
            this.out.close();

                        // Code to notify the end of the thread
        }
        catch(Exception e)
        {
            System.err.println("Hola!");
            System.err.println(e);
        }
    }
}

With this code if I execute only 1 thread, the encryption/decryption goes perfect, but with 2+ threads there is a problem with bytes in the zone between threads jobs, the data gets corrupted there and the checksum also fails.

I'm trying to do this with threads because it gets near 2x faster than with one thread, I think it should be because of processing and not by the accessing of the file.

As a irrelevant data, it compress 250Mb of data in 43 seconds on a MB Air. ¿It's a good time?

Answer 1

AESCrypt is not thread safe. You cannot use multiple threads with it.

Generally speaking, encryption code is rarely thread safe, as it requires complex mathematics to generate secure output. AES by itself is relatively fast, if you need better speed from it, consider vertical scaling or hardware accelerators as a first step. Later, you can add more servers to encrypt different files concurrently (horizontal scaling).

Answer 2

It makes absolutely no sense to use more than 1 thread for the HMAC because 1) it has to be computed sequentially and 2) I/O access R/W is much slower than actual HMAC computation

For AES it can be a good idea to use multiple threads when using CNT mode or other chaining modes which don't require knowledge of previous data blocks.

what about moving the question to crypto-stackexchange?

Answer 3

You basically want to multithread an operation that is intrinsically sequential.

Stream cipher cannot be made parallel because each block depends on the completion of the previous block. So you can encrypt multiple files in parallel independently with slight performance increase, especially if the files are in memory rather than on disk, but you cannot encrypt a single file using multiple cores.

As I can see, you use an update method. I'm not an expert in Java crypography but even the name of the method tells me that the encryption algorithm holds a state: "multithreading" and "state" are not friends, you have to deal with state management across threads.

Race condition explains why you get blocks damaged.