CODEWITHSUNDEEP
javascriptjavacryptographyrsawebcrypto-api
Lee Jeonghyun
Lee Jeonghyun

Reputation: 150

RSA decrypt failing between Java and browser using web crypto api

I'm trying to get encrypted response from server using RSA public key. Cipher is generated on server side but decoding on client side fails. Web crypto API throws DOM Exception.

Java server:

byte[] exponentBytes = Base64.getUrlDecoder().decode(body.exponent);
byte[] modulusBytes = Base64.getUrlDecoder().decode(body.modulus);

BigInteger exponent = new BigInteger(1, exponentBytes);
BigInteger modulus = new BigInteger(1, modulusBytes);

RSAPublicKeySpec spec = new RSAPublicKeySpec(modulus, exponent);
KeyFactory factory = KeyFactory.getInstance("RSA");
PublicKey publicKey = factory.generatePublic(spec);

Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
byte[] cipherBytes = cipher.doFinal("hello".getBytes())
return Base64.getEncoder().encodeToString(cipherBytes);

Browser:

const key = await window.crypto.subtle.generateKey(
  {
    name: 'RSA-OAEP',
    modulusLength: 512,
    publicExponent: new Uint8Array([1, 0, 1]),
    hash: 'SHA-256',
  },
  true,
  ['encrypt', 'decrypt'],
)

const jwk = await window.subtle.exportKey('jwk', key.publicKey);

const response = await fetch('/foo/bar', { method: 'post', body: { exponent: jwk.e, modulus: jwk.n } });

const body = await response.text();

const binary = window.atob(body);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) {
  bytes[i] = binary.charCodeAt(i);
}

await window.crypto.subtle.decrypt(
  { name: 'RSA-OAEP' },
  key.privateKey,
  bytes.buffer,
); // returns undefined throws the error

EDIT: After further research, I found that:

  1. (Obviously) cipher generated by web crypto api has no problem with decoding.
  2. Returning byte array itself from server side doesn't help.

Upvotes: 1

Views: 1123

Answers (1)

Michael Fehr
Michael Fehr

Reputation: 6424

On Webcrypto-side you are using

name: 'RSA-OAEP',
hash: 'SHA-256',
...

to instantiate the algorithm. On Java side you "just" instantiate the cipher with

Cipher cipher = Cipher.getInstance("RSA");

but that is the synonym for

Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1PADDING");

You need to use those lines to instantiate the Webcrypto algorithm:

    Cipher encryptCipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    OAEPParameterSpec oaepParameterSpecJCE = new OAEPParameterSpec("SHA-256", "MGF1", MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
    encryptCipher.init(Cipher.ENCRYPT_MODE, publicKey, oaepParameterSpecJCE);
    ciphertextByte = encryptCipher.doFinal(plaintextByte);

Security note: a keylength of 512 is UNSECURE, kindly use a minimum of 2048 bits key length.

Upvotes: 5

Related Questions