got AEADBadTagException while trying to decrypt message in GCM mode

Question

I'm writing an app which got a lot of security constraints: It needs to store files securely encrypted, and must be able to decrypt them. Also, an Operator needs to be able to decrypt the file without the app.

To archive this, I generated a KeyPair on my PC, put the public part in my app, generate an AES SecretKey Key inside the app, encrypt and save it with my public key (for operator purposes), and put the unencrypted key in AndroidKeyStore.

To Encrypt a message, I receive the SecretKey from KeyStore, encrypt my message, get the IV I used as well as the encryptedSecretKey, and write them in a defined order to a byte array (iv->encryptedSecretKey->encryptedMessage).

To Decrypt I try the same in reverse: get the byte array, read the iv and encryptedSecretKey, and pass the rest (encryptedMessage) to my cypher to decrypt. The problem is, that cipher.doFinal(encryptedMessage) is throwing an javax.crypto.AEADBadTagExceptionwhich is caused by android.security.KeyStoreException: Signature/MAC verification failed.

I already checked that the encrypted message and the one I want to decrypt are exactly the same. I'm having no idea what I am doing wrong.

The class I use is the following:

package my.company.domain;

import android.content.Context;
import android.content.SharedPreferences;
import android.security.keystore.KeyProperties;
import android.security.keystore.KeyProtection;
import android.support.annotation.NonNull;
import android.util.Base64;
import android.util.Log;

import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.security.GeneralSecurityException;
import java.security.InvalidKeyException;
import java.security.KeyFactory;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.NoSuchProviderException;
import java.security.PublicKey;
import java.security.SecureRandom;
import java.security.spec.X509EncodedKeySpec;

import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public class CryptoHelper {

    public static final String TAG = CryptoHelper.class.getSimpleName();

    private static final String KEY_ALIAS = "OI1lTI1lLI1l0";
    private static final char[] KEY_PASSWORD = "Il0VELI1lO".toCharArray();

    private static final String PREF_NAME = "CryptoPrefs";
    private static final String KEY_ENCRYPTED_SECRET = "encryptedSecret";

    private static final String ANDROID_KEY_STORE = "AndroidKeyStore";

    private static final int    IV_SIZE = 12;
    private static final int    IV_BIT_LEN = IV_SIZE * 8;


    //generate 128 bit key (16), other possible values 192(24), 256(32)
    private static final int    AES_KEY_SIZE = 16;
    private static final String AES = KeyProperties.KEY_ALGORITHM_AES;
    private static final String AES_MODE = AES + "/" + KeyProperties.BLOCK_MODE_GCM + "/" + KeyProperties.ENCRYPTION_PADDING_NONE;

    private static final String RSA = KeyProperties.KEY_ALGORITHM_RSA;
    private static final String RSA_MODE = KeyProperties.KEY_ALGORITHM_RSA + "/" + KeyProperties.BLOCK_MODE_ECB + "/" + KeyProperties.ENCRYPTION_PADDING_NONE;
    private static final String RSA_PROVIDER = "AndroidOpenSSL";

    private final Context mContext;
    private final SharedPreferences mPrefs;

    private SecureRandom mSecureRandom;
    private KeyStore mAndroidKeyStore;
    private PublicKey mPublicKey;
    private byte[] mEncryptedSecretKey;

    public CryptoHelper(Context context) {
        mContext = context;
        mSecureRandom = new SecureRandom();
        mPrefs = mContext.getSharedPreferences(PREF_NAME, Context.MODE_PRIVATE);
        try {
            mAndroidKeyStore = KeyStore.getInstance(ANDROID_KEY_STORE);
            mAndroidKeyStore.load(null);

        } catch (KeyStoreException e) {
            Log.wtf(TAG, "Could not get AndroidKeyStore!", e);
        } catch (Exception e) {
            Log.wtf(TAG, "Could not load AndroidKeyStore!", e);
        }
    }

    public void reset() throws KeyStoreException {
        mAndroidKeyStore.deleteEntry(KEY_ALIAS);
    }

    public byte[] encrypt(byte[] message){
        SecretKey secretKey = getSecretKey();
        try {
            Cipher cipher = Cipher.getInstance(AES_MODE);
            cipher.init(Cipher.ENCRYPT_MODE, getSecretKey());

            byte[] cryptedBytes = cipher.doFinal(message);
            byte[] iv = cipher.getIV();
            byte[] encryptedSecretKey = getEncryptedSecretKey();
            ByteBuffer buffer = ByteBuffer.allocate(IV_BIT_LEN + encryptedSecretKey.length + cryptedBytes.length);
            buffer
                .put(iv)
                .put(encryptedSecretKey)
                .put(cryptedBytes);
            return buffer.array();
        } catch (GeneralSecurityException e) {
            e.printStackTrace();
        }
        return null;
    }

    public byte[] encrypt(String message){
        return encrypt(message.getBytes(StandardCharsets.UTF_8));
    }

    public byte[] decrypt(byte[] bytes){
        ByteBuffer buffer = ByteBuffer.wrap(bytes);
        byte[] iv = new byte[IV_SIZE];
        buffer.get(iv);
        byte[] unused = getEncryptedSecretKey();
        buffer.get(unused);
        byte[] encryptedMessage = new byte[bytes.length - IV_SIZE - unused.length];
        buffer.get(encryptedMessage);
        try {
            Cipher cipher = Cipher.getInstance(AES_MODE);
            GCMParameterSpec parameterSpec = new GCMParameterSpec(IV_BIT_LEN, iv);
            cipher.init(Cipher.DECRYPT_MODE, getSecretKey(), parameterSpec);
            byte[] decryptedMessage = cipher.doFinal(encryptedMessage);
            return decryptedMessage;
        } catch (GeneralSecurityException e) {
            e.printStackTrace();
        }
        return null;
    }

    public String decryptToString(byte[] bytes){
        return new String(decrypt(bytes), StandardCharsets.UTF_8);
    }

    public byte[] decrypt(FileInputStream fileToDecrypt){
        byte[] buffer = null;
        try {
            buffer = new byte[fileToDecrypt.available()];
            fileToDecrypt.read(buffer);
            buffer = decrypt(buffer);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return buffer;
    }


    public PublicKey getPublicKey() {
        if (null == mPublicKey) {
            mPublicKey = readPublicKey();
        }
        return mPublicKey;
    }

    public byte[] getEncryptedSecretKey() {
        if (null == mEncryptedSecretKey){
            mEncryptedSecretKey = Base64.decode(mPrefs.getString(KEY_ENCRYPTED_SECRET, null), Base64.NO_WRAP);
        }
        return mEncryptedSecretKey;
    }

    private void saveEncryptedSecretKey(byte[] encryptedSecretKey){
        String base64EncryptedKey = Base64.encodeToString(encryptedSecretKey, Base64.NO_WRAP);
        mPrefs.edit().putString(KEY_ENCRYPTED_SECRET, base64EncryptedKey).apply();
    }

    protected SecretKey getSecretKey(){
        SecretKey secretKey = null;
        try {
            if (!mAndroidKeyStore.containsAlias(KEY_ALIAS)){
               generateAndStoreSecureKey();
            }
            secretKey = (SecretKey) mAndroidKeyStore.getKey(KEY_ALIAS, KEY_PASSWORD);
        } catch (KeyStoreException e) {
            Log.wtf(TAG, "Could not check AndroidKeyStore alias!", e);
            secretKey = null;
        } catch (GeneralSecurityException e) {
            e.printStackTrace();
            secretKey = null;
        }
        return secretKey;
    }

    private void generateAndStoreSecureKey()
            throws NoSuchPaddingException, NoSuchAlgorithmException, NoSuchProviderException, InvalidKeyException, KeyStoreException, BadPaddingException, IllegalBlockSizeException {
        SecretKey secretKey = generateSecureRandomKey();
        PublicKey publicKey = getPublicKey();
        Cipher keyCipher = Cipher.getInstance(RSA_MODE, RSA_PROVIDER);
        keyCipher.init(Cipher.ENCRYPT_MODE, publicKey);
        byte[] encryptedSecretKeyBytes = keyCipher.doFinal(secretKey.getEncoded());

        saveEncryptedSecretKey(encryptedSecretKeyBytes);

        KeyProtection keyProtection = new KeyProtection.Builder(KeyProperties.PURPOSE_DECRYPT | KeyProperties.PURPOSE_ENCRYPT | KeyProperties.PURPOSE_VERIFY)
                .setBlockModes(KeyProperties.BLOCK_MODE_GCM)
                .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE)
                .build();
        mAndroidKeyStore.setEntry(KEY_ALIAS, new KeyStore.SecretKeyEntry(secretKey), keyProtection);
    }


    protected PublicKey readPublicKey() {
        DataInputStream dis = null;
        PublicKey key = null;
        try {
            dis = new DataInputStream(mContext.getResources().getAssets().open("public_key.der"));
            byte[] keyBytes = new byte[dis.available()];
            dis.readFully(keyBytes);

            X509EncodedKeySpec spec = new X509EncodedKeySpec(keyBytes);
            KeyFactory facotory = KeyFactory.getInstance(RSA);
            key = facotory.generatePublic(spec);
        } catch (Exception e) {
            key = null;
        } finally {
            if (null != dis) {
                try {
                    dis.close();
                } catch (IOException e) {
                    Log.wtf(TAG, "Cannot Close Stream!", e);
                }
            }
        }
        return key;
    }

    @NonNull
    protected SecretKey generateSecureRandomKey() {
        return new SecretKeySpec(generateSecureRandomBytes(AES_KEY_SIZE), AES);
    }

    @NonNull
    protected byte[] generateSecureRandomBytes(int byteCount) {
        byte[] keyBytes = new byte[byteCount];
        mSecureRandom.nextBytes(keyBytes);
        return keyBytes;
    }
}

And I Test it like this:

@Test
public void testCrypto() throws Exception {
    CryptoHelper crypto = new CryptoHelper(InstrumentationRegistry.getTargetContext());
    crypto.reset();
    String verySecretOpinion = "we're all doomed";
    byte[] encrypt = crypto.encrypt(verySecretOpinion);
    Assert.assertNotNull("Encrypted secret is Null!", encrypt);
    Assert.assertFalse("encrypted Bytes are the same as Input!", new String(encrypt, StandardCharsets.UTF_8).equals(verySecretOpinion));
    String decryptedString = crypto.decryptToString(encrypt);
    Assert.assertNotNull("Decrypted String must be Non-Null!", decryptedString);
    Assert.assertEquals("Decrypted String doesn't equal encryption input!", verySecretOpinion, decryptedString);
}

By the way minSdkVersion is 25, so higher than Marshmallow

UPDATE:

1. Fixed Cipher.DECRYPT_MODE to ENCRYPT_MODE on saving the SecretKey thx to James K Polk's comment
2. it works If I switch from BlockMode GCM to BlockMode CBC (and changing the GCMParameterSpec to IvParamterSpec but loose the verification of the GCM Mode.

Answer 1

There are at least two problems with the Operator interface. First, you RSA encrypt the secret key using the wrong Cipher mode: you used DECRYPT mode when you should have used encrypt. Secondly, you are using RSA without any padding. You need to use a real padding mode, one of the OEAP padding modes is recommended.

An error in the encryption side occurs when sizing the buffer used to hold the result:

ByteBuffer buffer = ByteBuffer.allocate(IV_BIT_LEN + encryptedSecretKey.length + cryptedBytes.length);

allocates too much space. IV_BIT_LEN should probably be changed to IV_SIZE to get the correctly sized ByteBuffer.

The last mistake is the failure to account for the GCM authentication tag length when setting the GCMParameterSpec on the decrypt side. You initialized the tag length in this line

GCMParameterSpec parameterSpec = new GCMParameterSpec(IV_BIT_LEN, iv);

but that's incorrect, the tag length is unrelated to the IV. Since you did not explicitly set the GCMParameterSpec on the encrypt side you got the default tag length, which happens to be 128.

You can retrieve the tag length on the encrypt side by calling cipher.getParameters().getParameterSpec(GCMParameterSpec.class) to get the parameter spec. From this you can retrieve both the tag length and the iv. You should probably consider the tag length, 16 bytes = 128 bits, to be a hard-coded constant and not transmit it. The receiver should similar assume the tag length is 128 bits.