RSA / Asymmetric decryption in android

Question

Edit: Refactored the code based on comments, to validte Base64 as suggested

I am trying to decode an RSA encrypted string on in Android, unsuccessfully.

// Retrieve the key from server
// Looks counter intuitive, but, the retrieval happens on trusted Intranet.
URL keyUrl = new URL(keyUri);
URLConnection keyConn = keyUrl.openConnection();
InputStream keyStream = keyConn.getInputStream();

// Create the keystore and load the key
KeyStore pkcsStore = KeyStore.getInstance("PKCS12");
pkcsStore.load(keyStream, "password".toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("X509");
kmf.init(pkcsStore, "password".toCharArray());

// Retrieve the key from keystore
String clientKeyAlias = pkcsStore.aliases().nextElement();
Key clientKey = pkcsStore.getKey(clientKeyAlias, "password".toCharArray());
Log.d(TAG, String.format("Algorithm: %s Format: %s ", clientKey.getAlgorithm(), clientKey.getFormat()));

// Retrieve the public key from keystore
Certificate certificate = pkcsStore.getCertificate(clientKeyAlias);
PublicKey clientPublicKey = certificate.getPublicKey();

// Let's make sure the certificate/key stored in pkcsStore are valid
String plainText = "Stackoverflow is Amazing!";
Log.d(TAG,  String.format("Encrypting string '%s' to validate cert/key pair stored in pkcs store", plainText));
byte[] plainTextBytes = plainText.getBytes("UTF-8");

// Let's get the cipher instance
Cipher testCipher = Cipher.getInstance(clientKey.getAlgorithm());
testCipher.init(Cipher.ENCRYPT_MODE, clientPublicKey);

byte[] encryptedTextBytes = testCipher.doFinal(plainTextBytes);

testCipher.init(Cipher.DECRYPT_MODE, clientKey);
byte[] decryptedTextBytes = testCipher.doFinal(encryptedTextBytes);

String decryptedPlainText = new String(decryptedTextBytes, "UTF-8");
Log.d(TAG, "Decrypted plainText " + decryptedPlainText);

// Retrieve the encrypted data from server
URL symUrl = new URL(symUri);
URLConnection connection = symUrl.openConnection();
int contentLength = connection.getContentLength();
Log.d(TAG, "Content length " + contentLength);

InputStream is = connection.getInputStream();

BufferedReader br = new BufferedReader(new InputStreamReader(is));
// Validate the base64 string matches the data on server
String payLoad = br.readLine().trim(); // Trim the line in case echo there is a newline character
String base64StringOnServer = "nMdLvh+PDH1DimjQV0kTYU9E5tJA9VA2ADlcCQVkYwYZQWRFdT57bW4oDvlpGJ1KzJ/N3smyV5N7xlpNJo6HXans+cCImZI5MaZ9prEc3gUtIlwDWj80e0L+5YaRSIshXjPKvurjhwmppz+IAuh5Aq/c3ZdE2nbFn+L+2Ih91nsl6WV1diCEBw6pUIMtdPh8nXmQfp3ZbDKHt3VnSkoLLhPXYbYaTOaTuhAJZhszE5ejpB0sQ78d/1WbmPU1MnGCrUSOkM6s22fT3aTyzxo3IjmzXBWhz32tCbOBagmAsyaHlLHbQ2pQoxI9cR0hcZAc/IyEFKrVSJnFA03P/QzkhQ==";
Log.d(TAG, "Base64 as stored on the server: " + base64StringOnServer);
Log.d(TAG, "Base64 received from server: " + payLoad);
if (base64StringOnServer.equals(payLoad)) {
    Log.d(TAG, "Base64 string comparision passed");
}

// Get the byte array of payload
byte[] data = Base64.decode(payLoad, Base64.DEFAULT);

// Decipher the data
Cipher cipher = Cipher.getInstance(clientKey.getAlgorithm());
cipher.init(Cipher.DECRYPT_MODE, clientKey);
byte[] decryptedKeyBytes = cipher.doFinal(data);

// Validate the decrypted data
String decryptedKey = new String (decryptedKeyBytes, "UTF-8");
Log.d(TAG, "Decrypted Key: " + decryptedKey);

Here is the openssl command encrypting the data

openssl rsautl -encrypt -pubin -inkey rsakey.pub -in key.txt | base64 -w 0 > enc_base64.txt

I have decrypted payload by using private key at the source to validate as seen below.

cat key.base64 | base64 --decode |  openssl rsautl -decrypt -inkey rsakey.key

The code show above runs fine with no exceptions, but the decrypted payload is not valid.

Here is the excerpt from Logcat

// Base64 decode flag Base64.NO_WRAP
Algorithm: RSA Format: PKCS#8 
Encrypting string 'Stackoverflow is Amazing!' to validate cert/key pair stored in pkcs store
Decrypted plainText Stackoverflow is Amazing!
Content length 344
Base64 as stored on the server: nMdLvh+PDH1DimjQV0kTYU9E5tJA9VA2ADlcCQVkYwYZQWRFdT57bW4oDvlpGJ1KzJ/N3smyV5N7xlpNJo6HXans+cCImZI5MaZ9prEc3gUtIlwDWj80e0L+5YaRSIshXjPKvurjhwmppz+IAuh5Aq/c3ZdE2nbFn+L+2Ih91nsl6WV1diCEBw6pUIMtdPh8nXmQfp3ZbDKHt3VnSkoLLhPXYbYaTOaTuhAJZhszE5ejpB0sQ78d/1WbmPU1MnGCrUSOkM6s22fT3aTyzxo3IjmzXBWhz32tCbOBagmAsyaHlLHbQ2pQoxI9cR0hcZAc/IyEFKrVSJnFA03P/QzkhQ==
Base64 received from server: nMdLvh+PDH1DimjQV0kTYU9E5tJA9VA2ADlcCQVkYwYZQWRFdT57bW4oDvlpGJ1KzJ/N3smyV5N7xlpNJo6HXans+cCImZI5MaZ9prEc3gUtIlwDWj80e0L+5YaRSIshXjPKvurjhwmppz+IAuh5Aq/c3ZdE2nbFn+L+2Ih91nsl6WV1diCEBw6pUIMtdPh8nXmQfp3ZbDKHt3VnSkoLLhPXYbYaTOaTuhAJZhszE5ejpB0sQ78d/1WbmPU1MnGCrUSOkM6s22fT3aTyzxo3IjmzXBWhz32tCbOBagmAsyaHlLHbQ2pQoxI9cR0hcZAc/IyEFKrVSJnFA03P/QzkhQ==
Base64 string comparision passed
Decrypted Key: )|~`!��u��   zU.�縉)���#9�]�׶��Q����݌�Ő�A�f*(A�%U�/;�%2��j0I$dI*,'@�E�t@�P�=-C��VR�o06p�4op9�k�q  >�qq�>V�~Y���R�O���t2�S�֖E���&�rP�Q��L����2�;7��}�g�Wpٟ$P�N0�k�w��ň1����0�2��2�jҗ/�}r:*g"ոm���[����1234567812345678

// Base64 decode flag Base64.DEFAULT
Algorithm: RSA Format: PKCS#8 
Encrypting string 'Stackoverflow is Amazing!' to validate cert/key pair stored in pkcs store
Decrypted plainText Stackoverflow is Amazing!
Content length 344
Base64 as stored on the server: nMdLvh+PDH1DimjQV0kTYU9E5tJA9VA2ADlcCQVkYwYZQWRFdT57bW4oDvlpGJ1KzJ/N3smyV5N7xlpNJo6HXans+cCImZI5MaZ9prEc3gUtIlwDWj80e0L+5YaRSIshXjPKvurjhwmppz+IAuh5Aq/c3ZdE2nbFn+L+2Ih91nsl6WV1diCEBw6pUIMtdPh8nXmQfp3ZbDKHt3VnSkoLLhPXYbYaTOaTuhAJZhszE5ejpB0sQ78d/1WbmPU1MnGCrUSOkM6s22fT3aTyzxo3IjmzXBWhz32tCbOBagmAsyaHlLHbQ2pQoxI9cR0hcZAc/IyEFKrVSJnFA03P/QzkhQ==
Base64 received from server: nMdLvh+PDH1DimjQV0kTYU9E5tJA9VA2ADlcCQVkYwYZQWRFdT57bW4oDvlpGJ1KzJ/N3smyV5N7xlpNJo6HXans+cCImZI5MaZ9prEc3gUtIlwDWj80e0L+5YaRSIshXjPKvurjhwmppz+IAuh5Aq/c3ZdE2nbFn+L+2Ih91nsl6WV1diCEBw6pUIMtdPh8nXmQfp3ZbDKHt3VnSkoLLhPXYbYaTOaTuhAJZhszE5ejpB0sQ78d/1WbmPU1MnGCrUSOkM6s22fT3aTyzxo3IjmzXBWhz32tCbOBagmAsyaHlLHbQ2pQoxI9cR0hcZAc/IyEFKrVSJnFA03P/QzkhQ==
Base64 string comparision passed
Decrypted Key: )|~`!��u��   zU.�縉)���#9�]�׶��Q����݌�Ő�A�f*(A�%U�/;�%2��j0I$dI*,'@�E�t@�P�=-C��VR�o06p�4op9�k�q  >�qq�>V�~Y���R�O���t2�S�֖E���&�rP�Q��L����2�;7��}�g�Wpٟ$P�N0�k�w��ň1����0�2��2�jҗ/�}r:*g"ոm���[����1234567812345678

Answer 1

There are some missing details, but one bug that immediately jumps out is this:

Cipher cipher = Cipher.getInstance(clientKey.getAlgorithm());

Key.getAlgorithm() does not return enough information to be used as an argument to Cipher.getInstance(). It might return without throwing an exception, but the result will not be correct. Instead, always specify the full transformation string "algorithm/mode/padding" as the argument to Cipher.getInstance(). If you really encrypted the data as you indicated with openssl then this would be the correct line:

Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");