Asymmetric encryption discrepancy - Android vs Java

Question

I've recently started writing an Android version of an online game that I've written in Java. However, I am running into an inconsistency with encryption. The java app works fine - it reads in the public key from a file, encrypts some text and passes it to the server where it is decrypted correctly using the private key. On android, everything seems to work (and is running through the same code), but the Server has a BadPaddingException trying to decrypt the message. I've included all the relevant code and step-by-step sequence of events below:

The first thing that happens upon connecting to the server is the agreement of the symmetric key. This is generated on the client, thus:

SecretKey symmetricKey = null;
try
{
    KeyGenerator keyGen = KeyGenerator.getInstance("AES");
    symmetricKey = keyGen.generateKey();
}
catch (Throwable t)
{
    Debug.stackTrace(t, "Failed to generate symmetric key.");
}

return symmetricKey;

It is then converted to a Base64 string:

byte[] keyBytes = secretKey.getEncoded();
return base64Interface.encode(keyBytes);

And encrypted using the public key:

public static String encrypt(String messageString, Key key)
{
    String encryptedString = null;
    try
    {
        byte[] messageBytes = messageString.getBytes();
        String algorithm = key.getAlgorithm()
        Cipher cipher = Cipher.getInstance(algorithm);
        cipher.init(Cipher.ENCRYPT_MODE, key);
        byte[] cipherData = cipher.doFinal(messageBytes);
        encryptedString = base64Interface.encode(cipherData);

        //Strip out any newline characters
        encryptedString = encryptedString.replaceAll("\n", "");
        encryptedString = encryptedString.replaceAll("\r", "");
    }
    catch (Throwable t)
    {
        Debug.append("Caught " + t + " trying to encrypt message: " + messageString);
    }

    return encryptedString;
}

In this form, it is passed to the server which uses the private key to decrypt the message and recover the SecretKey object:

public static String decrypt(String encryptedMessage, Key key)
{
    String messageString = null;
    try
    {
        byte[] cipherData = base64Interface.decode(encryptedMessage);
        String algorithm = key.getAlgorithm();
        Cipher cipher = Cipher.getInstance(algorithm);
        cipher.init(Cipher.DECRYPT_MODE, key);
        byte[] messageBytes = cipher.doFinal(cipherData);
        messageString = new String(messageBytes);
    }
    catch (Throwable t)
    {
        Debug.append("Caught " + t + " trying to decrypt message: " + encryptedMessage, failedDecryptionLogging);
    }

    return messageString;
}

However, whenever I do this on a message passsed up from the Android app, the doFinal line produces the following exception:

11/07 12:55:55.975   javax.crypto.BadPaddingException: Decryption error
    at sun.security.rsa.RSAPadding.unpadV15(Unknown Source)
    at sun.security.rsa.RSAPadding.unpad(Unknown Source)
    at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:354)
    at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:380)
    at javax.crypto.Cipher.doFinal(Cipher.java:2121)
    at util.EncryptionUtil.decrypt(EncryptionUtil.java:85)
    at server.MessageHandlerRunnable.handleUnencryptedMessage(MessageHandlerRunnable.java:226)
    at server.MessageHandlerRunnable.getResponse(MessageHandlerRunnable.java:188)
    at server.MessageHandlerRunnable.run(MessageHandlerRunnable.java:85)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

My first thought was that the problem had to be in the Base64 encoding/decoding as that's the only bit of the code that's different between the Android and desktop versions. However, I've done some testing and verified that these are consistent and that my server code can recover the original text using either encoding method.

My next thought was that the Android version must somehow be using the wrong Public Key. This is generated on start-up from a file which is common to both platforms using the following code:

public static void generatePublicKey()
{
    InputStream in = null;
    ObjectInputStream oin = null;

    try
    {
        in = KeyGeneratorUtil.class.getResourceAsStream("/assets/public.key");
        oin = new ObjectInputStream(new BufferedInputStream(in));

        BigInteger m = (BigInteger) oin.readObject();
        BigInteger e = (BigInteger) oin.readObject();
        RSAPublicKeySpec keySpec = new RSAPublicKeySpec(m, e);
        KeyFactory fact = KeyFactory.getInstance("RSA");
        MessageUtil.publicKey = fact.generatePublic(keySpec);
    } 
    catch (Throwable e) 
    {
        Debug.stackTrace(e, "Unable to read public key - won't be able to communicate with Server.");
    } 
    finally
    {
        if (in != null)
        {
            try {in.close();} catch (Throwable t) {}
        }

        if (oin != null)
        {
            try {oin.close();} catch (Throwable t) {}
        }
    }
}

When I look at the key on the two platforms (using toString()), I see the following (I've truncated the moduli):

Desktop:

Sun RSA public key, 1024 bits modulus: 11920225567195913955197820411061866681846853580... public exponent: 65537

Android:

OpenSSLRSAPublicKey{modulus=a9bfe8d8a199fc6a... ,publicExponent=10001}

At first glance these appear to be completely different, however I'm now convinced that they are equivalent with one expressed in decimal (Desktop) and the other in hex (Android). 10001 in hex is equivalent to 65537 in decimal, and putting the hex modulus into an online converter produces a number that at least starts with the correct digits for the decimal modulus. Why, then, am I seeing the BadPaddingException?

One final thing worth noting, this seems to be the same issue that was raised in this question about a year ago:

RSA on Android is different from PC

However, no solution was presented and I thought it worth making a new question where I could present all the information that I have available to me.

Answer 1

BadPaddingExceptions are commonly caused by one of the following:

1. The ciphertext that is passed to the decryption function does not equal the ciphertext received from the encryption function (just a single bit difference will completely break the process).

2. A different key (or a non-matching private key) is used for decrypting the data.

3. The program tries to unpad a message padded with scheme A using scheme B.

Since you have already verified 1. and 2. are correct, 3. is most likely the root your problem.

You are using the same code in the same programming language on both ends, so why should there be any incompatibilities? The culprits are the following two lines of code:

String algorithm = key.getAlgorithm();
Cipher cipher = Cipher.getInstance(algorithm);

Since a key can be used in combination with any padding scheme, it only saves the algorithm it is for, which has the effect that key.getAlgorithm() returns "RSA". Calling Cipher.getInstance("RSA") is generally not a good idea because Java will automatically choose platform dependent defaults for the cipher-mode and the padding.

This can be avoided by passing full cipher-strings ("<algorithm>/<mode>/<padding>") like for example "RSA/ECB/PKCS1Padding". It is always a good idea to do this for all Cipher instances when using the JCE library.

Answer 2

If you want to do RSA encryption/decryption using Public-Private key pair, here is a code sample that works in Android as well as any server/desktop Java program

It has three methods:

1. generateRSAKeyPair - This generates public private key pair. You will have to store public key on the app and private key on your server.
2. encryptRSA - This method encrypts the given data using public key. It returns byte array that represents encrypted value. If you need, you can do base 64 encoding of this value so that you can use it in JSON
3. decryptRSA - This decrypts the encrypted value using private key. Make sure that you pass raw encrypted value to this method, and not a base 64 encoded value.

.

import java.security.Key;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.SecureRandom;
import javax.crypto.Cipher;

public class Test {

    private static final String RSA_ECB_PKCS1_PADDING = "RSA/ECB/PKCS1Padding";

    public static void main(String[] args) {
        String data = "Hello World";

        KeyPair kp = generateRSAKeyPair();

        PublicKey publicKey = kp.getPublic();
        PrivateKey privateKey = kp.getPrivate();

        byte[] encryptedValue = encryptRSA(publicKey, data.getBytes());
        byte[] decrytpedValue = decryptRSA(privateKey, encryptedValue);

        String decryptedData = new String(decrytpedValue);

        System.out.println(decryptedData);
    }

    public static KeyPair generateRSAKeyPair() {
        KeyPairGenerator keyGen;
        try {
            keyGen = KeyPairGenerator.getInstance("RSA");
            SecureRandom rnd = new SecureRandom();
            keyGen.initialize(2048, rnd);
            KeyPair keyPair = keyGen.genKeyPair();
            return keyPair;
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
            return null;
        }
    }

    public static byte[] encryptRSA(Key key, byte[] data) {
        byte[] cipherText = null;
        try {
            final Cipher cipher = Cipher.getInstance(RSA_ECB_PKCS1_PADDING);
            cipher.init(Cipher.ENCRYPT_MODE, key);
            cipherText = cipher.doFinal(data);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return cipherText;
    }

    public static byte[] decryptRSA(Key key, byte[] data) {
        byte[] decryptedText = null;
        try {
            final Cipher cipher = Cipher.getInstance(RSA_ECB_PKCS1_PADDING);
            cipher.init(Cipher.DECRYPT_MODE, key);
            decryptedText = cipher.doFinal(data);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return decryptedText;
    }

}