CODEWITHSUNDEEP
c#compression
Missy
Missy

Reputation: 1368

How to Compress Large Files C#

I am using this method to compress files and it works great until I get to a file that is 2.4 GB then it gives me an overflow error:

 void CompressThis (string inFile, string compressedFileName)
 {

        FileStream sourceFile = File.OpenRead(inFile);
        FileStream destinationFile = File.Create(compressedFileName);


        byte[] buffer = new byte[sourceFile.Length];
        sourceFile.Read(buffer, 0, buffer.Length);

        using (GZipStream output = new GZipStream(destinationFile,
            CompressionMode.Compress))
        {
            output.Write(buffer, 0, buffer.Length);
        }

        // Close the files.
        sourceFile.Close();
        destinationFile.Close();
  }

What can I do to compress huge files?

Upvotes: 4

Views: 11550

Answers (3)

Vadim Martynov
Vadim Martynov

Reputation: 8902

You should not to write the whole file to into the memory. Use Stream.CopyTo instead. This method reads the bytes from the current stream and writes them to another stream using a specified buffer size (81920 bytes by default).

Also you don't need to close Stream objects if use using keyword.

void CompressThis (string inFile, string compressedFileName)
{
    using (FileStream sourceFile = File.OpenRead(inFile))
    using (FileStream destinationFile = File.Create(compressedFileName))
    using (GZipStream output = new GZipStream(destinationFile, CompressionMode.Compress))
    {
        sourceFile.CopyTo(output);
    }
}

You can find a more complete example on Microsoft Docs (formerly MSDN).

Upvotes: 10

Abhishek Dhotre
Abhishek Dhotre

Reputation: 511

Alternative solution for zip format without allocating memory -

using (var sourceFileStream = new FileStream(this.GetFilePath(sourceFileName), FileMode.Open))
            {
                using (var destinationStream =
                    new FileStream(this.GetFilePath(zipFileName), FileMode.Create, FileAccess.ReadWrite))
                {
                    using (var archive = new ZipArchive(destinationStream, ZipArchiveMode.Create, true))
                    {
                        var file = archive.CreateEntry(sourceFileName, CompressionLevel.Optimal);
                        using (var entryStream = file.Open())
                        {
                            var fileStream = sourceFileStream;
                            await fileStream.CopyTo(entryStream);
                        }
                    }
                }
            }

The solution will write directly from input stream to output stream

Upvotes: 3

Adam Brown
Adam Brown

Reputation: 1729

You're trying to allocate all of this into memory. That just isn't necessary, you can feed the input stream directly into the output stream.

Upvotes: 4

Related Questions