Trying to decrypt a string using openssl/golang which has been encrypted in rails

Question

I am trying to decrypt a string which has been encrypted in my rails project. This is how I am encrypting the data:

def encrypt_text(text_To_encrypt)
        # 0. generate the key using command openssl rand -hex 16 on linux machines
        # 1. Read the secret from config
        # 2. Read the salt from config
        # 3. Encrypt the data
        # 4. return the encypted data
        # Ref: http://www.monkeyandcrow.com/blog/reading_rails_how_does_message_encryptor_work/
        secret = Rails.configuration.miscconfig['encryption_key']
        salt = Rails.configuration.miscconfig['encryption_salt']
        key = ActiveSupport::KeyGenerator.new(secret).generate_key(salt, 32)
        crypt = ActiveSupport::MessageEncryptor.new(key)
        encrypted_data = crypt.encrypt_and_sign(text_To_encrypt)
        encrypted_data
end

Now the issue is I am not able to decrypt it using openssl. It just shows bad magic number. Once I do that in open ssl, my plan is to decrypt it in golang.

Here is how I tried to descrypt it using openssl:

openssl enc -d -aes-256-cbc -salt -in encrypted.txt -out decrypted.txt -d -pass pass:<the key given in rails> -a

This just shows bad magic number

Answer 1

Trying to decrypt data encrypted in a different system will not work unless you are aware and deal with the many intricate details of how both systems do the cryptography. Although both Rails and the openssl command line tool use the OpenSSL libraries under the hood for their crypto operations, they both use it in their own distinct ways that are not directly interoperable.

If you look close to the two systems, you'll see that for example:

• Rails message encryptor not only encrypts the message but also signs it
• Rails encryptor uses Marshal to serialize the input data
• the openssl enc tool expects the encrypted data in a distinct file format with a Salted__<salt> header (this is why you get the bad magic number message from openssl)
• the openssl tool must be properly configured to use the same ciphers as Rails encryptor and key generator, as openssl defaults are different from Rails defaults
• the default ciphers configuration changed significantly since Rails 5.2.

With this general info, we can have a look at a a practical example. It is tested in Rails 4.2 but should work equally up to Rails 5.1.

Anatomy of a Rails-encrypted message

Let me start with a slightly amended code that you presented. The only changes there are to preset the password and salt to static values and print a lot of debug info:

def encrypt_text(text_to_encrypt)
  password = "password" # the password to derive the key
  salt = "saltsalt" # salt must be 8 bytes

  key = ActiveSupport::KeyGenerator.new(password).generate_key(salt, 32)

  puts "salt (hexa) = #{salt.unpack('H*').first}" # print the saltin HEX
  puts "key (hexa) = #{key.unpack('H*').first}" # print the generated key in HEX

  crypt = ActiveSupport::MessageEncryptor.new(key)
  output = crypt.encrypt_and_sign(text_to_encrypt)
  puts "output (base64) = #{output}"
  output
end

encrypt_text("secret text")

When you run this, you'll get something like the following output:

salt (hexa) = 73616c7473616c74
key (hexa) = 196827b250431e911310f5dbc82d395782837b7ae56230dce24e497cf07b6518
output (base64) = SGRTUXYxRys1N1haVWNpVWxxWTdCMHlyMk15SnQ0dWFBOCt3Z0djWVdBZz0tLTkrd1hBNWJMVm9HcnptZ3loOG1mNHc9PQ==--80d091e8799776113b2c0efd1bf75b344bf39994

The last line (output of the encrypt_and_sign method) is a combination of two parts separated by -- (see source):

1. the encrypted message (Base64-encoded) and
2. the message signature (Base64-encoded).

The signature is not important for encryption so let's take a look in the first part - let's decode it in Rails console:

> Base64.strict_decode64("SGRTUXYxRys1N1haVWNpVWxxWTdCMHlyMk15SnQ0dWFBOCt3Z0djWVdBZz0tLTkrd1hBNWJMVm9HcnptZ3loOG1mNHc9PQ==")
=> "HdSQv1G+57XZUciUlqY7B0yr2MyJt4uaA8+wgGcYWAg=--9+wXA5bLVoGrzmgyh8mf4w=="

You can see that the decoded message again consists of two Base64-encoded parts separated by -- (see source):

1. the encrypted message itself
2. the initialization vector used in the encryption

Rails message encryptor uses the aes-256-cbc cipher by default (note that this has changed since Rails 5.2). This cipher needs an initialization vector, which is randomly generated by Rails and must be present in the encrypted output so that we can use it together with the key to decipher the message.

Moreover, Rails does not encrypt the input data as a simple plain text, but rather a serialized version of the data, using the Marshal serializer by default (source). If we decrypted such serialized value with openssl, we would still get a slightly garbled (serialized) version of the initial plain text data. That's why it will be more appropriate to disable serialization while encrypting the data in Rails. This can be done by passing a parameter to the encryption method:

  # crypt = ActiveSupport::MessageEncryptor.new(key)
  crypt = ActiveSupport::MessageEncryptor.new(key, serializer: ActiveSupport::MessageEncryptor::NullSerializer)

A re-run of the code yields output that is slightly shorter than the previous version, because the encrypted data has not been serialized now:

salt (hexa) = 73616c7473616c74
key (hexa) = 196827b250431e911310f5dbc82d395782837b7ae56230dce24e497cf07b6518
output (base64) = SUlIWFBjSXRUc0JodEMzLzhXckJzUT09LS1oZGtPV1ZRc2I5Wi8zOG01dFNOdVdBPT0=--58bbaf983fd20459062df8b6c59eb470311cbca9

Finally, we must find out some info about the encryption key derivation procedure. The source tells us that the KeyGenerator uses the pbkdf2_hmac_sha1 algorithm with 2**16 = 65536 iterations to derive the key from the password / secret.

Anatomy of an openssl encrypted message

Now, a similar investigation is needed on the openssl side to learn the details of its decryption process. First, if you encrypt anything using the openssl enc tool, you will find out that the output has a distinct format:

Salted__<salt><encrypted_message>

It begins with the Salted__ magic string, then followed by the salt (in hex form) and finally followed by the encrypted data. To be able to decrypt any data using this tool, we must get our encrypted data into the same format.

The openssl tool uses the EVP_BytesToKey (see source) to derive the key by default but can be configured to use the pbkdf2_hmac_sha1 algorithm using the -pbkdf2 and -md sha1 options. The number of iterations can be set using the -iter option.

How to decrypt Rails-encrypted message in openssl

So, finally we have enough information to actually try to decrypt a Rails-encrypted message in openssl.

First we must decode the first part of the Rails-encrypted output again to get the encrypted data and the initialization vector:

> Base64.strict_decode64("SUlIWFBjSXRUc0JodEMzLzhXckJzUT09LS1oZGtPV1ZRc2I5Wi8zOG01dFNOdVdBPT0=")
=> "IIHXPcItTsBhtC3/8WrBsQ==--hdkOWVQsb9Z/38m5tSNuWA=="

Now let's take the IV (the second part) and convert it to a hexa string form, as that is the form that openssl needs:

> Base64.strict_decode64("hdkOWVQsb9Z/38m5tSNuWA==").unpack("H*").first
=> "85d90e59542c6fd67fdfc9b9b5236e58"  # the initialization vector in hex form

Now we need to take the Rails-encrypted data and convert it to the format that openssl will recognize, i.e. prepend it with the magic string and salt and Base64-encode it again:

> Base64.strict_encode64("Salted__" + "saltsalt" + Base64.strict_decode64("IIHXPcItTsBhtC3/8WrBsQ=="))
=> "U2FsdGVkX19zYWx0c2FsdCCB1z3CLU7AYbQt//FqwbE=" # encrypted data suitable for openssl

Finally, we can construct the openssl command to decrypt the data:

$ echo  "U2FsdGVkX19zYWx0c2FsdCCB1z3CLU7AYbQt//FqwbE=" | 
> openssl enc -aes-256-cbc -d -iv 85d90e59542c6fd67fdfc9b9b5236e58 \
>   -pass pass:password -pbkdf2 -iter 65536 -md sha1 -a
secret text

And voilá, we successfully decrypted the initial message!

The openssl parameters are as follows:

• -aes-256-cbc sets the same cipher as Rails uses for encryption
• -d stands for decryption
• -iv passes the initialization vector in the hex string form
• -pass pass:password sets the password used to derive the encryption key to "password"
• -pbkdf2 and -md sha1 set the same key derivation algorithm as is used by Rails (pbkdf2_hmac_sha1)
• -iter 65536 sets the same number of iterations for key derivation as was done in Rails
• -a allows to work with Base64-encoded encrypted data - no need to handle raw bytes in files

By default openssl reads from STDIN, so we simply pass the encrypted data (in proper format) to openssl using echo.

debugging

In case you hit any problems when decrypting with openssl, it is useful to add the -P parameter to the command line, which outputs debugging info about the cipher / key parameters:

$ echo ... | openssl ... -P
salt=73616C7473616C74
key=196827B250431E911310F5DBC82D395782837B7AE56230DCE24E497CF07B6518
iv =85D90E59542C6FD67FDFC9B9B5236E58

The salt, key, and iv values must correspond to the debugging values printed by the original code in the encrypt_text method printed above. If they are different, you know you are doing something wrong...

Now, I guess you can expect similar problems when trying to decrypt the message in go but I think you have some good pointers now to start.