CODEWITHSUNDEEP
javaencryptionaessamlshibboleth
jn1kk
jn1kk

Reputation: 5112

SAML RSA & AES Decryption - Random Junk Bytes At End

SAML XML responses are encrypted by our Gluu/Shibboleth server with my public cert. I have read the spec, and with the help of Stackoverflow, implemented the solution. However, upon decryption, I get random characters at the end.

SAML responses use an RSA-ECB/MGF1 encoded AES-128-CBC key. So first I have to decode the AES key (bytes) and then use that AES key to decrypt the XML response.

Here is my code:

public static void main(String[] args) throws Exception {
    Path p = Paths.get("C:\\Users\\jj\\Desktop\\myPrivateKey.key");
    String encryptedAESKey = "FUZLPtkLSUgOo0bETQ5hwP1OWNggGlWhG+Z......wF1G6twRjg=="; // from XML
    byte[] aesKey = decryptWithPem("RSA/ECB/OAEPwithSHA1andMGF1Padding", "RSA", Util.base64DecodeAsBytes(encryptedAESKey), p);
    String encryptedXML = "YfJu7h4Id09hpuoqthl3Ks/JqhIXm.....amb24JZu7cJZT3cEO2a2U6qi0VCyoXQ=";
    byte[] decryptedData = decrypt("AES/CBC/NoPadding", "AES", Util.base64DecodeAsBytes(encryptedXML), aesKey);
    for(int i = decryptedData.length - 20; i < decryptedData.length; i++) {
        System.out.println("i: " + i + " -> " + decryptedData[i]); // print last 20 bytes
    }
    System.out.println(new String(decryptedData)); // prints <saml2:Assertion xmlns:saml2="urn:oasis:names:........</saml2:Assertion>�G{A

}

Note that random bytes on the print statement! Last line prints:

<saml2:Assertion xmlns:saml2="urn:oasis:names:........</saml2:Assertion>�G{A

I realized that the first 16 bytes in the message are the IV, so I strip them from the message (getting rid of the junk from the beginning of the message). But now I get random 5 bytes at the end of the message. Those bytes are:

i: 1931 -> -120
i: 1932 -> 71
i: 1933 -> 123
i: 1934 -> 65
i: 1935 -> 5

Other functions:

public static byte[] decryptWithPem(String alg, String pemAlg, byte[] encryptedData, Path pemPath) {
    try {
        Cipher cipher = Cipher.getInstance(alg, "BC");
        cipher.init(Cipher.DECRYPT_MODE, loadPrivateKey(pemPath, pemAlg));
        return cipher.doFinal(encryptedData);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

private static PrivateKey loadPrivateKey(Path keyPath, String alg) {
    try {
        byte[] keyData = Util.base64DecodeAsBytes(IOUtil.fileToString(keyPath).replaceAll("\\s", ""));
        KeyFactory keyFactory = KeyFactory.getInstance(alg);
        EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(keyData);
        return keyFactory.generatePrivate(privateKeySpec);
    } catch(NoSuchAlgorithmException | InvalidKeySpecException e) {
        throw new RuntimeException(e);
    }
}

private static SecretKeySpec getSecretKeySpec(String alg, byte[] key) {
    return new SecretKeySpec(key, alg);
}

public static byte[] decrypt(String alg, String keyAlg, byte[] dataToDecrypt, byte[] key) {
    try {
        Cipher cipher = Cipher.getInstance(alg, "BC");
        cipher.init(Cipher.DECRYPT_MODE, getSecretKeySpec(keyAlg, key), new IvParameterSpec(dataToDecrypt, 0, 16));
        return cipher.doFinal(Arrays.copyOfRange(dataToDecrypt, 16, dataToDecrypt.length));
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

I am using bouncy castle. If I use PKCS7 padding, I get an error about erroneous padding.

The way the AES key is encrypted: http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p, http://www.w3.org/2000/09/xmldsig#sha1. The way XML data is encrypted: http://www.w3.org/2001/04/xmlenc#aes128-cbc.

Is it possible the message is randomly padded?

------ EDIT ------

It seems that the spec uses ISO 10126 padding, use "AES/CBC/ISO10126Padding" instead of "AES/CBC/NoPadding".

Upvotes: 4

Views: 1284

Answers (1)

Artjom B.
Artjom B.

Reputation: 61902

Is it possible the message is randomly padded?

Yes, as detailed in the Padding section it uses random padding by saying that the question marks can be of any value, but the last byte denotes the length of the padding. Their example: 0x2122232425262728??????????????08.

This is actually ISO 10126 padding and you can easily remove it by looking at the last byte:

byte[] pp = cipher.doFinal(Arrays.copyOfRange(dataToDecrypt, 16, dataToDecrypt.length));
return Arrays.copyOf(pp, pp.length - pp[pp.length-1]);

Note that if you're handling the padding yourself, you must use NoPadding in alg.

Upvotes: 5

Related Questions