CODEWITHSUNDEEP
c#filememoryencryptionaes
javaseaayameradost
javaseaayameradost

Reputation: 119

Decrease memory consumption when encrypting files c#

My encryption app is using to much memory and it simply can't handle large files, how can I optimize my code for handling large files? I am using the code below to convert the file to base64 (increasing the file size dramatically)

Console.Write("Enter File Path: ");
docPath = Console.ReadLine();
extension = docPath.Substring(docPath.IndexOf(".")).Trim();
byte[] binarydata = File.ReadAllBytes(docPath);
text = System.Convert.ToBase64String(binarydata, 0, binarydata.Length);
var Encrypted = AESCryptography.Encrypt(text, m.ToString(), extension);
using (FileStream fs = File.Create(docPath.Substring(0,docPath.IndexOf(".")) + ".aent"))
{
    Byte[] info = new UTF8Encoding(true).GetBytes(Encrypted);
    // Add some information to the file.
    fs.Write(info, 0, info.Length);
}

How can I do this in blocks? Heres my encryption class:

public static class AESCryptography
{
    private const int keysize = 256;
    public static string Encrypt(string plainText, string passPhrase, string extention)
    {
        byte[] plainTextBytes = Encoding.UTF8.GetBytes(plainText);
        using (PasswordDeriveBytes password = new PasswordDeriveBytes(passPhrase, null))
        {
            byte[] keyBytes = password.GetBytes(keysize / 8);
            using (RijndaelManaged symmetricKey = new RijndaelManaged())
            {
                symmetricKey.GenerateIV();
                symmetricKey.Mode = CipherMode.CBC;
                using (ICryptoTransform encryptor = symmetricKey.CreateEncryptor(keyBytes, symmetricKey.IV))
                {
                    using (MemoryStream memoryStream = new MemoryStream())
                    {
                        using (CryptoStream cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
                        {
                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
                            cryptoStream.FlushFinalBlock();
                            byte[] cipherTextBytes = memoryStream.ToArray();
                            return Convert.ToBase64String(cipherTextBytes) + "\n" + Convert.ToBase64String(symmetricKey.IV) + "\n" + extention;
                        }
                    }
                }
            }
        }
    }

    public static string Decrypt(string cipherText, string passPhrase, string initVector)
    {
        byte[] initVectorBytes = Convert.FromBase64String(initVector);
        byte[] cipherTextBytes = Convert.FromBase64String(cipherText);
        using (PasswordDeriveBytes password = new PasswordDeriveBytes(passPhrase, null))
        {
            byte[] keyBytes = password.GetBytes(keysize / 8);
            using (RijndaelManaged symmetricKey = new RijndaelManaged())
            {
                symmetricKey.Mode = CipherMode.CBC;
                using (ICryptoTransform decryptor = symmetricKey.CreateDecryptor(keyBytes, initVectorBytes))
                {
                    using (MemoryStream memoryStream = new MemoryStream(cipherTextBytes))
                    {
                        using (CryptoStream cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
                        {
                            byte[] plainTextBytes = new byte[cipherTextBytes.Length];
                            int decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
                        }
                    }
                }
            }
        }
    }
}

Upvotes: 1

Views: 1034

Answers (2)

Remus Rusanu
Remus Rusanu

Reputation: 294267

  • Don't store the IV at the end of the file. You are forcing a reader to read the entire encrypted file first just to discover the IV needed top read the file. Store it at the beginning of the file.

  • Don't encode the file base64. Base64 is a transmission format, not a storage one.

  • Use Stream.CopyTo to read the file and write into the CryptoStream w/o reading all in memory at once. Back the CryptoStream by the destination file stream, not by a memory stream.

  • Ditto for decryption, read the CryptoStream (backed by input stream, after skipping the IV) and CopyTo the destination file stream

Upvotes: 2

ken2k
ken2k

Reputation: 48985

  • First you're reading bytes from your file: byte[] binarydata = File.ReadAllBytes(docPath);
  • Then you're converting bytes to a string: text = System.Convert.ToBase64String(binarydata, 0, binarydata.Length);
  • Then you're getting your bytes back from the string: byte[] plainTextBytes = Encoding.UTF8.GetBytes(plainText);

You're currently storing in memory all of your large file twice (actually way more than twice because base64 string uses more memory than just a byte array).

Solution:

  • Drop the part that converts to/from string so you don't use strings at all but only bytes.
  • Then drop the whole ReadAllBytes thing that loads all your file in memory and process by small chunks instead. See Using Rijndael encryption for large files for example

Upvotes: 3

Related Questions