CODEWITHSUNDEEP
javascriptjavaencryptionrsa
Andy King
Andy King

Reputation: 1662

RSA Encryption with OAEP between Java and JavaScript

I am trying to encrypt a short string in JavaScript and decrypt it in Java. The decryption fails, and I think it's because of a difference in the block mode and/or padding between the two platforms. I tried encrypting the same string in both Java and JavaScript, and got different results, which indicates that there is indeed a difference. Here is the Java code that creates the key:

KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(1024);
KeyPair keyPair = kpg.generateKeyPair();

And here is the Java code that I used to test the encryption:

Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.ENCRYPT_MODE, keyPair.getPublic());
byte[] bytes = cipher.doFinal("asdf".getBytes());

I send the public key across to the JavaScript process, and convert it to an ArrayBuffer, with the variable name publicKey. I have verified that the key on the JavaScript side matches the key on the Java side (by exporting it with crypto.subtle.exportKey and examining the bytes). Here is the JavaScript code that I used to test the encryption:

crypto.subtle.importKey('spki', publicKey,
                        {hash: 'SHA-256', name: 'RSA-OAEP'}, true,
                        ['encrypt'])
      .then((key) => {
        crypto.subtle.encrypt({name: 'RSA-OAEP'}, key,
                              new TextEncoder().encode('asdf'))
              .then((buffer) => {

              });
      });

The contents of the byte array in Java and the array buffer in JavaScript are not the same. The settings that I am unsure of are the parameter to Cipher#getInstance on the Java side and the parameters to importKey and encrypt on the JavaScript side. Are there any settings that will work between Java and JavaScript using the built-in classes? Or should I look at third-party libraries (e.g., Bouncy Castle)?

Upvotes: 5

Views: 19962

Answers (3)

Float
Float

Reputation: 61

In addition to @Chip's answer - which was really helpful - I would like to add the following case:

Assume you want to use the following for decryption in Javascript (webcrypto):

window.crypto.subtle.decrypt(
        {
            name: "RSA-OAEP",
            hash: { name: "SHA-512" }
            //label: Uint8Array([...]) //optional
        },
        privateRsaKey, //CryptoKey object containing private RSA key
        encdata //ArrayBuffer containing to be decrypted data
    )
    .catch(function(err){
        console.error(err);
    })

Then you have to use the following OAEPParameterSpec for encryption in Java (and likely vice versa but I did not try that): 

OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-512", "MGF1", 
        new MGF1ParameterSpec("SHA-512"), PSource.PSpecified.DEFAULT);

Since @Chip referred to the MGF1 Padding I only, I assumed one would have to use 

OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-512", "MGF1", 
        new MGF1ParameterSpec("SHA-256"), PSource.PSpecified.DEFAULT);

but apparently one has to change both hash functions to SHA-512 as shown in my first OAEPParameterSpec code block.

Upvotes: 6

Chip
Chip

Reputation: 3316

This is old, but here's an alternate solution if you want to use subtle crypto in javascript, and have control over the java decryption.

Here's how you decrypt in Java assuming you used the original JS code in the question to encrypt:

Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPPadding");
OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-256"), PSource.PSpecified.DEFAULT);
cipher.init(Cipher.DECRYPT_MODE, privKey, oaepParams);
byte[] decrypted = cipher.doFinal(encryptedBytes)

The issue with the Cipher RSA/ECB/OAEPWithSHA-256AndMGF1Padding is that it uses SHA-1 for the MGF1 Padding by default. Javascript uses SHA-256, which causes the mismatch. By specifying the MGF1ParamterSpec, we can force Java to use the same hashing algorithm as Javascript default.

Upvotes: 22

Andy King
Andy King

Reputation: 1662

It looks as though the built-in encryption/decryption in JavaScript and Java do not have compatible settings for RSA encryption. A viable solution appears to be the forge library from github (forge on github). The key settings are described on the github page as follows (RSA examples):

// encrypt data with a public key using RSAES-OAEP/SHA-256/MGF1-SHA-1
// compatible with Java's RSA/ECB/OAEPWithSHA-256AndMGF1Padding
var encrypted = publicKey.encrypt(bytes, 'RSA-OAEP', {
  md: forge.md.sha256.create(),
  mgf1: {
    md: forge.md.sha1.create()
  }
});

Upvotes: 0

Related Questions