Java 7 : output from encryption with public RSA key using cipher "RSA/ECB/OAEPWithSHA1AndMGF1Padding" does not match with openssl command

Question

We have to encrypt our data with HMAC-SHA256 that needs randomly generated salt. We are generating the salt this way:

 public String generateSalt() throws Exception
    {
        KeyGenerator keyGen;
        String salt = null;
        try
        {
            keyGen = KeyGenerator.getInstance( "HmacSHA256" );
            keyGen.init( 128 );
            SecretKey key = keyGen.generateKey();

            byte[] encodedKey = key.getEncoded();
            salt = Base64.encodeBase64String( key.getEncoded() );
            LOG.info( "Salt : " + salt );

        }
        catch ( NoSuchAlgorithmException  )
        {
            e.printStackTrace();
            throw e;
        }

        return salt;
    }

According to our test this salt generation part is right. I have issue with the next part:

Now I have to write this salt in binary format in a file ( say named as pie_raw) and that's been done as:

    private void writeToFile( byte[] saltBytes, String fileName ) throws FileNotFoundException, IOException
    {

        DataOutputStream out = new DataOutputStream( new FileOutputStream( enviro.getOutputFilePath()
                + fileName ) );

        out.write( saltBytes );
        out.close();
        LOG.info( " Raw file created : " + enviro.getOutputFilePath() + fileName );
    }

And then, I have to encrypt this salt with a supplied public RSA key in ".pem" and for Java implementation, the cipher will be "RSA/ECB/OAEPWithSHA1AndMGF1Padding". And finally the binary ciphertext should be written to a file named "pie_key". This part has been implemented this way:

    private byte[] encryptSalt( String salt ) throws Exception
    {
        byte[] cipheredKey = null;
        try
        {

             String keyString= readKeyFile( enviro.getPublicKeyFile() );
             byte[] pem = pemToDer(keyString); 
             PublicKey publicKey = derToPublicKey(pem);


            //PublicKey publicKey = getPublicKey( enviro.getPublicKeyFile() );
            // Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());

            Cipher rsaCipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");
            rsaCipher.init( Cipher.ENCRYPT_MODE, publicKey );
            cipheredKey = rsaCipher.doFinal( salt.getBytes( "UTF-8" ) );//"UTF-8"
            LOG.info( "Cyphered key : " + cipheredKey.toString() );

        }
        catch ( IOException | GeneralSecurityException e )
        {
            e.printStackTrace();
            throw e;
        }

        return cipheredKey;
    }

    static String readKeyFile( String path )
            throws IOException
    {
        String line = null;

        try (BufferedReader br =
                new BufferedReader( new FileReader( path ) ))
        {

            StringBuilder sb = new StringBuilder();
            line = br.readLine();

            while ( line != null )
            {
                sb.append( line );
                sb.append( "\n" );
                line = br.readLine();
            }

            return sb.toString();
        }

    }

    public static byte[] pemToDer( String pemKey ) throws GeneralSecurityException
    {
        String[] parts = pemKey.split( "-----" );
        return DatatypeConverter.parseBase64Binary( parts[ parts.length / 2 ] );
    }

    public static PublicKey derToPublicKey( byte[] asn1key ) throws GeneralSecurityException
    {
        X509EncodedKeySpec spec = new X509EncodedKeySpec( asn1key );
        KeyFactory keyFactory = KeyFactory.getInstance( "RSA" );
        return keyFactory.generatePublic( spec );
    }

Writing this encrypted salt to a file named as "pie_key" in binary format by calling the "writeToFile" method above.

Now the content of the file "pie_key" should match the out put of the cmd :

openssl rsautl -encrypt -pubin -inkey wrap_pie_key_rsa.pem -oaep -in pie_key.raw -out pie_key

But whatever I tried ( you may find some signs of the ways, I tried ) did not work means that the final binary-encrypted-salt did not match with the output of openssl cmd.

Any idea what I am doing wrong?

I am using Java 7. And the .pem (partial) looks like

-----BEGIN PUBLIC KEY-----
MIIBIjANBgk345iG9w0BAQEFAA54328AMIIBCgKCAQEAt4GLJGPmvYdxwwAe59n3
. 
.
.
.
7QIDNQAB
-----END PUBLIC KEY-----

Thanks in advance.

Answer 1

First of all, as Artjom already mentioned, the padding for OAEP or PKCS#1 v1.5 compatible padding is randomized. So even if you encrypt the same salt multiple times you would not get the same value. You can only decrypt the result to see if encryption succeeded.

---

Furthermore, you say you need a binary salt, but you first encode the salt to base64. It's unlikely that your encryption should contain an encoded salt. Maybe you need to encode the output of the encryption, not the salt.

The spurious encoding happens in the following line:

salt = Base64.encodeBase64String( key.getEncoded() );

---

Finally, although a new HMAC key generally consists of fully random bytes, I would say that it is not the right way to generate a salt. Instead just use:

SecureRandom rngForSalt = new SecureRandom();
byte[] salt = new byte[SALT_SIZE];
rngForSalt.nextBytes(salt);

---

Note too that the Bouncy Castle lightweight API (i.e. calling org.bouncycastle functionality directly) contains a PEM codec. No need to program or hack that yourself.

---

Try this Java 8 code. Bouncy castle provider classes required (no need to register the provider, this is just for the PEM handling).

package nl.maartenbodewes.stackoverflow;

import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.file.Files;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.KeyFactory;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.interfaces.RSAPublicKey;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;

import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.KeyGenerator;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;

import org.bouncycastle.util.io.pem.PemObject;
import org.bouncycastle.util.io.pem.PemReader;
import org.bouncycastle.util.io.pem.PemWriter;

public class GenerateAndWrapHMACKey {

    public static SecretKey generateHMACKey() throws Exception {
        final KeyGenerator keyGen;
        try {
            keyGen = KeyGenerator.getInstance("HmacSHA256");
        } catch (NoSuchAlgorithmException e) {
            throw new RuntimeException("HMAC KeyGeneration should be available");
        }
        keyGen.init(128);
        SecretKey key = keyGen.generateKey();
        return key;
    }

    public static void writeToFile(SecretKey key, String filename)
            throws IOException {
        // file handling probably should be in a separate class
        Files.write((new File(filename)).toPath(), key.getEncoded());
    }

    public static RSAPublicKey readRSAPublicKey(String filename) throws IOException, InvalidKeySpecException {
        try (PemReader reader = new PemReader(new FileReader(filename))) {
            PemObject pemObject = reader.readPemObject();
            KeyFactory kf;
            try {
                kf = KeyFactory.getInstance("RSA");
            } catch (NoSuchAlgorithmException e) {
                throw new RuntimeException("RSA key factory not available", e);
            }
            KeySpec keySpec = new X509EncodedKeySpec(pemObject.getContent());
            try {
                return (RSAPublicKey) kf.generatePublic(keySpec);
            } catch (ClassCastException e) {
                throw new InvalidKeySpecException("That's no RSA key", e);
            }
        }
    }

    public static byte[] wrapKey(Key key, RSAPublicKey wrappingKey) throws InvalidKeyException, IllegalBlockSizeException {
        Cipher rsaWrapper;
        try {
            rsaWrapper = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");
            rsaWrapper.init(Cipher.WRAP_MODE, wrappingKey);
        } catch (NoSuchAlgorithmException | NoSuchPaddingException | InvalidKeyException e) {
            throw new RuntimeException("RSA OAEP should be available for RSA public key", e);
        }
        return rsaWrapper.wrap(key);
    }

    public static void main(String[] args) throws Exception {
        // we need an RSA PEM key first I guess :)
        KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
        kpg.initialize(1024, new SecureRandom());
        KeyPair kp = kpg.generateKeyPair();
        String publicKeyFilename = "rsa_pub.pem";
        try (PemWriter pemWriter = new PemWriter(new FileWriter(publicKeyFilename))) {
            pemWriter.writeObject(new PemObject("PUBLIC KEY", kp.getPublic().getEncoded()));
        }

        RSAPublicKey wrappingRSAPublicKey = readRSAPublicKey(publicKeyFilename);
        SecretKey hmacKey = generateHMACKey();
        byte[] wrappedKey = wrapKey(hmacKey, wrappingRSAPublicKey);
        System.out.println(Base64.getEncoder().encodeToString(wrappedKey));
    }
}