AES/GCM PBKDF2 Encryption for Kotlin and Dart

Question

I am attempting to use GCM encryption with PBKDF2 that is interoperable across both kotlin and dart. Decrypters will come next. Currently I am using a "working" kotlin version (below) and I want to replicate it in dart (my attempt below that) if it is correct. See below Kotlin version (Notice log results provided below respective lines. Outputs are suspect.):

Also note: These examples now use the same text and masterpass inputs.

KOTLIN:

fun encrypt(input: String, password: String): String {
val masterpw = getKey(password).toString(Charset.forName("UTF-8"))

val mastertest = getKey(password)

val random = SecureRandom()
Log.d("RANDOM", "${random}") //D/RANDOM  (25834): java.security.SecureRandom@dc72083

val salt = ByteArray(8)
Log.d("SALT", "${salt}") //D/SALT    (25834): [B@202f200

random.nextBytes(salt)
Log.d("SALT2", "${salt}") //D/SALT2   (25834): [B@202f200

val factory: SecretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1")
Log.d("factory", "${factory}") //D/factory (25834): javax.crypto.SecretKeyFactory@cfd3a39

val spec: KeySpec = PBEKeySpec(masterpw.toString().toCharArray(), salt, 10000, 256)
Log.d("KeySpec", "${spec}") //D/KeySpec (25834): javax.crypto.spec.PBEKeySpec@a5587e

val tmp: SecretKey = factory.generateSecret(spec)
Log.d("SecretKey", "${tmp}") //D/SecretKey(25834): com.android.org.bouncycastle.jcajce.provider.symmetric.util.BCPBEKey@6d141df

val iv = ByteArray(12)
Log.d("IV", "${iv}") //D/IV      (25834): [B@aa52e2c

random.nextBytes(iv)
Log.d("IV2", "${iv}") //D/IV2     (25834): [B@aa52e2c

val cipher = Cipher.getInstance("AES/GCM/NoPadding")
Log.d("Cipher", "${cipher}") //D/Cipher  (25834): javax.crypto.Cipher@e6e40f5

cipher.init(Cipher.ENCRYPT_MODE, tmp, IvParameterSpec(iv))
Log.d("Cipher2", "${cipher}") //D/Cipher2 (25834): javax.crypto.Cipher@e6e40f5

val cipherText: ByteArray = cipher.doFinal(input.toByteArray(Charset.forName("UTF-8")))
Log.d("cipherText", "${cipherText}") //D/cipherText(25834): [B@f3a7e8a

val ivstring: String = Base64.encodeToString(iv, Base64.NO_WRAP)
Log.d("ivstring", "${ivstring}") //D/ivstring(25834): D3tPtM6+WYnoSswE

val saltystring: String = Base64.encodeToString(salt, Base64.NO_WRAP)
Log.d("saltystring", "${saltystring}") //D/saltystring(25834): zbq9ZqJ9xiw=

val cipherstring: String = Base64.encodeToString(cipherText, Base64.NO_WRAP)
Log.d("cipherstring", "${cipherstring}") //D/cipherstring(25834): w/WqSqg++udXCLKE6ly765OWBHKt79Lw/g==

val returnstring: String = ivstring + "-" + saltystring + "-" + cipherstring
Log.d("returnstring", "${returnstring}") //D/returnstring(25834): D3tPtM6+WYnoSswE-zbq9ZqJ9xiw=-w/WqSqg++udXCLKE6ly765OWBHKt79Lw/g==

return returnstring
}

fun getKey(masterPass: String): ByteArray {
    return masterPass.padEnd(32, '.').toByteArray(Charset.forName("UTF-8"))
}

DART:

The dart method utilizes the cryptography.dart package v1.4.1. Note: Due to constraints with other libraries in the app, I can't use a newer version of the cryptography package, which I believe rules the recently added 'AesGcm.with128bits' functions out. The dart version crashes near the end when I attempt to decode the encrypted cipherTextBytes to a string, as shown in the provided log results.

encryptPassGCM(String text, String masterPass) async {
print("ENCRYPTPASSGCM STARTS WITH: " + "TEXT: " + text + " & " + "master: " + masterPass);

//trim key and convert
String keyString = masterPass;
if (keyString.length < 32) {
  int count = 32 - keyString.length;
  for (var i = 0; i < count; i++) {
    keyString += ".";
  }
}

Uint8List keyStringutf8  = utf8.encode(keyString);
print("keyStringutf8: " + keyStringutf8.toString());

//gen salt and iv
final salt = Nonce.randomBytes(8);
print("salt init: " + salt.bytes.toString());
//LOG salt init: [161, 50, 222, 98, 151, 225, 89, 65]
final iv = Nonce.randomBytes(12);
print("iv init: " + iv.bytes.toString());
//LOG iv init: [59, 188, 146, 172, 213, 13, 135, 35, 202, 220, 178, 190]

//create key
final pbkdf2 = Pbkdf2(
 macAlgorithm: Hmac(sha1),
 iterations: 10000,
  bits: 256,
);
final keyBytes = await pbkdf2.deriveBits(
  keyStringutf8,
  nonce: salt,
);

print("keybytes: " + keyBytes.toString());
//LOG keybytes: [85, 204, 96, 108, 200, 21, 24, 115, 254, 104, 133, 81, 53, 126, 252, 161, 172, 193, 25, 177, 143, 69, 53, 35, 105, 144, 248, 6, 121, 106, 237, 142]
SecretKey secretKey = new SecretKey(keyBytes);
print("secretKey: " + secretKey.toString());
//LOG secretKey: SecretKey(...)

//create ciphertext and convert to string
List<int> textutf8 = utf8.encode(text);
Uint8List cipherTextBytes = await AesGcm().encrypt(textutf8, secretKey: secretKey, nonce: iv); 
print("cipherTextBytes: " + cipherTextBytes.toString()); 
//LOG cipherTextBytes: [110, 3, 238, 169, 52, 125, 176, 200, 122, 142, 111, 75, 181, 248, 91, 57, 95, 131, 85, 223, 224, 73, 173, 39, 37]
var cipherText = utf8.decode(cipherTextBytes); //CRASHES HERE

print("cipherText: " + cipherText);
var cipherString = iv.toString() + "-" + salt.toString() + "-" + cipherText;
print("GCM CIPHER STRING COMPLETE: " + cipherString);
return cipherString;
}

1. Is the kotlin implementation correct? The data logs seem erroneous.
2. What is wrong with my dart implementation?

Answer 1

The exception is thrown when trying to Utf8 decode the ciphertext. Arbitrary binary data like ciphertexts or pseudo-random data like salts or IVs cannot be decoded with charset encodings like Utf8 because the data will be corrupted, see here. Instead, a binary-to-text encoding like Base64 must be applied.

The Kotlin code Base64 encodes salt, IV and ciphertext and concatenates the portions with a separator (-). The Dart counterpart would be e.g.:

var cipherString = base64.encode(iv.bytes) + "-" + base64.encode(salt.bytes) + "-" + base64.encode(cipherTextBytes);

and the UTF8 decoding of cipherTextBytes is to be removed.

A second problem was that by accident the originally posted code did not use the key derived using PBKDF2, but a randomly generated key (see also Rob Napier's comment).

When both bugs are fixed, both codes are functionally identical and produce the same ciphertext (assuming the same salt and IV).

---

Note that the GCM authentication tag (16 bytes by default) is automatically appended to the ciphertext in the Dart and Kotlin code (i.e. cipherTextBytes contains not only the ciphertext but also the authentication tag: ciphertext|tag). Not all libraries do it this way, so the separation of the tag (as the last 16 bytes) is required when decrypting with such a library.

Also, it is rather uncommon to Base64 encode salt, IV and ciphertext/tag separately and concatenate the parts with a separator. Instead, by convention, concatenation is done at byte level: salt|IV|ciphertext|tag. Since the salt, IV and tag lengths are known on both sides, no separator is needed. The result is Base64 encoded. But as mentioned, this is just a convention.

Regarding the parameters for PBKDF2: Although SHA1 is classified as insecure (see here), its use as HMAC/SHA1 is not critical. However, a move to SHA256 would support the banishment of SHA1 from the ecosystem. The iteration count should slow down an attacker and should therefore be chosen as high as possible while maintaining acceptable performance (typically larger than 10,000). The recommended salt length is 16 bytes, see PBKDF2.