Encrypt by "DES-EDE3-CBC" in ruby and decrypt in Java

Question

I am trying to encrypt data by the cipher "DES-EDE3-CBC" in Ruby, then decrypt the encrypted data in Java.

Here is the code in Ruby I do the encrypting:

require 'digest'
require 'openssl'
require 'base64'

ALG = "DES-EDE3-CBC"
key = "80f28a1ef4aa9df6ee2ee3210316b98f383eb344"
cipher = OpenSSL::Cipher::Cipher.new(ALG)
cipher.pkcs5_keyivgen(key, nil)
cipher.encrypt
data = "hello"
result = cipher.update(data)
result << cipher.final
# Write the data to file.
File.open("enc.txt", "wb"){|f| f.write result}

Then decrypt in Java:

import java.security.*;
import java.*;
import java.io.*;
import javax.crypto.*;
import javax.crypto.spec.*;

public class Test{
  public static void main(String[] args) throws Exception {
    String key = "80f28a1ef4aa9df6ee2ee3210316b98f383eb344";

    // Init the key
    DESKeySpec desKeySpec = new DESKeySpec(key.getBytes());
    SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");
    Key secretKey = keyFactory.generateSecret(desKeySpec);

    Cipher cipher = Cipher.getInstance("DES/ECB/PKCS5Padding");
    cipher.init(Cipher.DECRYPT_MODE, secretKey);

    byte[] buf = new byte[1024];
    InputStream input = new FileInputStream(new File("enc.txt"));
    FileOutputStream output = new FileOutputStream(new File("dec.txt"));

    int count = input.read(buf);

    // Read and decrypt file content
    while (count >= 0) {
        output.write(cipher.update(buf, 0, count)); 
        count = input.read(buf);        
    }
    output.write(cipher.doFinal());
    output.flush();

  }
}

But I always get the exception when running the Java code:

Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded
  at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:811)
  at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:676)
  at com.sun.crypto.provider.DESCipher.engineDoFinal(DESCipher.java:314)
  at javax.crypto.Cipher.doFinal(Cipher.java:1969)
  at Test.main(Test.java:29)

I think the problem is the "DES-EDE3-CBC" cipher in Ruby is not compatible with Java "DES/ECB/PKCS5Padding". How I can do it?

Answer 1

On the Ruby side, don't use Cipher#key_ivgen unless it's a strict requirement. It has been deprecated and OpenSSL::PKCS5 should be used instead for password-based encryption. Your key seems to be hex, 40 characters, which results in at most 20 bytes of entropy. Any reason why it has this particular form? Triple DES key are 24 bytes long, the easiest way to create a secure key is to do:

cipher = OpenSSL::Cipher.new("DES-EDE3-CBC")
cipher.encrypt
key = cipher.random_key

When doing so, you should also always generate a random IV before encryption:

iv = cipher.random_iv

In order to communicate the key and IV to Java, if you should require a hex representation of them:

hex_key = key.unpack("H*")[0]

In Java, you made the mistake to use DES instead of DESede, and you use it in ECB mode, when in Ruby you were using CBC. Don't use ECB. You will also have to initialize the Cipher with the key and IV that were used in Ruby. You'll need a proper hex en-/decoder for this (search on the web), I've taken the first code sample I could find, but be warned that this is not the most efficient way to do it, a table-based lookup will be much faster. But we need something to get you started, so here it is:

public static byte[] hexDecode(String hex) {
    ByteArrayOutputStream bas = new ByteArrayOutputStream();
    for (int i = 0; i < hex.length(); i+=2) {
        int b = Integer.parseInt(hex.substring(i, i + 2), 16);
        bas.write(b);
    }
    return bas.toByteArray();
}

byte[] key = hexDecode("<hex representation of the Ruby key>");
byte[] iv = hexDecode("<hex representation of the Ruby IV>");

DESedeKeySpec desKeySpec = new DESedeKeySpec(key);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DESede");
Key secretKey = keyFactory.generateSecret(desKeySpec);
IvParameterSpec ivSpec = new IvParameterSpec(iv);
Cipher cipher = Cipher.getInstance("DESede/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, secretKey, ivSpec);
... /* do the decryption */