Java AES Encryption not decrypting correctly

Question

I have this test below where I'm trying to verify that my AES encryption/decryption with compression/decompression is working correctly. I have a test for just the encryption/decryption and a test for just the compression/decompression and I know individually they work correctly, but for some reason when I combine them the IV gets screwed up after the decompression. I'm stuck and don't know what else to look for, can anyone offer any suggestions or help? If I need to post more code let me know. I think I included the important stuff.

For context: What I'm trying to do is take some sensitive data, encrypt it with AES, encrypt the AES key with a RSA public key and then compress those two pieces of data so they can be sent over the wire. Then on the other end I want to decompress the data, use a RSA private key to decrypt the AES key and then use that to decrypt the data. If there is another way to accomplish this I'm not locked into writing everything my self. If you suggest a library please only suggest libraries that I can use in a commercial product.

@Test
public void testEncryptionDecryptionProcesses() throws SSHException {

    SSHKey key1 = generateKeyPair();

    UnencryptedData data = new UnencryptedData();
    data.setUserName("hardy");
    data.setHashedPassword("somepassword");
    data.setRequest("eat");
    data.setResponse("");
    data.setTimestamp(new Timestamp(new Date().getTime()).toString());
    data.setPublicKeyMod(key1.getPublicMod().toString());
    data.setPublicKey(key1.getPublicKey().toString());

    byte[] bytes = encryptAndCompress(data, key1);

    assertTrue(bytes != null);
    assertTrue(bytes.length > 0);

    UnencryptedData decryptedData = decompressAndDecrypt(bytes, key1);
    assertEquals(data.getDataForEncryption(), decryptedData.getDataForEncryption());

}

public static byte[] encryptAndCompress(UnencryptedData data, SSHKey sshKey) {

    byte[] results = null;

    try {
        byte[] aesKey = createKeyForAES(AES_BIT_LENGTH);
        //this should use the servers public key so that only the server can decrypt it
        //gather data, get a digest, encrypt the data
        UnencryptedData digestedData = createDigest(data);
        //encrypt it
        EncryptedData toCompress = encryptDataAES(digestedData, aesKey);
        String encryptedAESKey = encryptKey(sshKey, aesKey);
        toCompress.setEncryptedAESKey(encryptedAESKey);
        //compress it
        byte[] compressed = compressString(toCompress.getDataForCompression());
        //return the compressed and encrypted data.
        results = compressed;
    } catch(SSHException e) {

        Log.e("SSHFunctions.encryption", "Unable to run the encryption/compression process on the data");
    } catch (UnsupportedEncodingException e) {

        Log.e("SSHFunctions.encryption", "Charset not supported");
    } 

    return results;
}

public static UnencryptedData decompressAndDecrypt(byte[] data, SSHKey sshKey) {

    UnencryptedData results = null;

    try {
        //take the data and decompress it, should result in encryptedData|encryptedAESKey
        byte[] decompressed = decompressString(data);
        String decompressedStr = new String(decompressed, CHAR_SET);
        String[] decompressedArr = decompressedStr.split(SPLIT_STRING);
        //using the users private key decrypt the data
        byte[] decryptedKey = decryptKey(sshKey, decompressedArr[1]);
        EncryptedData encryptedData = new EncryptedData();
        encryptedData.setAesEncryptedData(decompressedArr[0].getBytes(CHAR_SET));
        encryptedData.setIV(decompressedArr[2].getBytes(CHAR_SET)); //TODO: this doesn't seem to decompress correctly
        //create a digest from the decrypted data and compare it with the digest that was included.
        UnencryptedData decryptedDate = decryptDataAES(encryptedData, decryptedKey);
        if(validDigest(decryptedDate)) {

            results = decryptedDate;
        }
        //if equal return the data, if not equal return null
    } catch(Exception e) {

        Log.e("SSHFunctions.decryption", "Unable to run the uncompress/decrypt process on the data");
    }

    return results;
}

public static byte[] decompressString(byte[] toDecompress) {

    ByteArrayInputStream bis = new ByteArrayInputStream(toDecompress);
    byte[] uncompressed;

    try {

        ByteArrayOutputStream buffer = new ByteArrayOutputStream();
        GZIPInputStream is = new GZIPInputStream(bis);

        byte[] tmp = new byte[256];

        while (true) {

            int r = is.read(tmp);
            if (r < 0) {

                break;
            }
            buffer.write(tmp, 0, r);
        }
        is.close();

        uncompressed = buffer.toByteArray();

        try {
            bis.close();
        } catch (IOException e) {
            ;
        }

        try {
            buffer.close();
        } catch (IOException e) {
            ;
        }
    } catch(IOException e) {

        uncompressed = null;
        Log.e("Zipfunctions.decompress", "Unable to decompress");
    }

    return uncompressed;    
}

public static byte[] compressString(byte[] toCompress) {

    byte[] toCompressBytes = toCompress;
    byte[] compressed;

    ByteArrayOutputStream bos = new ByteArrayOutputStream(toCompressBytes.length);
    try {
        GZIPOutputStream compressor = new GZIPOutputStream(bos);

        compressor.write(toCompressBytes, 0, toCompress.length);
        compressor.close();

        compressed = bos.toByteArray();

        try {
            bos.close();
        } catch(IOException e) {
            ;
        }
    } catch(IOException e) {

        compressed = null;
        Log.e("ZipFunctions.compress", "Unable to compress data");
    }

    return compressed;      
}

Answer 1

You're attempting to implement a crypto-system that is similar to OpenPGP. That is, you are wanting to encrypt arbitrary amounts of data with a symmetric key and securely share this key (and the encrypted data) with the recipient(s).

As a result, I would suggest you consider using the Java OpenPGP libraries provided by the BouncyCastle team.

Their license is very permissive. However, their documentation is infamously poor, so you will need to Google many examples to see how to achieve your goals.

Answer 2

First of all, you should not use string-delimiter to separate parts of byte buffer. It would cause problem if your delimiter character exists in your data.

I suggest you to read about TLV structure!

Here is a simple implementation of transport data!

class TransportData {
private byte[] iv;
private byte[] aesKey;
private byte[] encryptedData;

public TransportData() {
}

public TransportData(byte[] iv, byte[] aesKey, byte[] encryptedData) {
this.iv = iv;
this.aesKey = aesKey;
this.encryptedData = encryptedData;
}

public byte[] encode() throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
dos.writeInt(iv.length);
dos.write(iv);
dos.writeInt(aesKey.length);
dos.write(aesKey);
dos.writeInt(encryptedData.length);
dos.write(encryptedData);
dos.close();
return baos.toByteArray();
}

public void decode(byte[] buffer) throws IOException {
ByteArrayInputStream bais = new ByteArrayInputStream(buffer);
DataInputStream dis = new DataInputStream(bais);
iv = new byte[dis.readInt()];
dis.read(iv);
aesKey = new byte[dis.readInt()];
dis.read(aesKey);
encryptedData = new byte[dis.readInt()];
dis.read(encryptedData);
dis.close();
}

public byte[] getAesKey() {
return aesKey;
}
public byte[] getEncryptedData() {
return encryptedData;
}
public byte[] getIv() {
return iv;
}
}