CODEWITHSUNDEEP
c#jsongzipgzipstream
user3296355
user3296355

Reputation:

Decompress 100GB gzip JSON file in C#

Is there a way we can decompress 3 GB gzip file which contains 100GB JSON data to local drive using c#. I have found the following code to decompress but it has a limit of 8GB

public static void Decompress(FileInfo fileToDecompress)
   {
       using (FileStream originalFileStream = fileToDecompress.OpenRead())
       {
           string currentFileName = fileToDecompress.FullName;
           string newFileName = currentFileName.Remove(currentFileName.Length - fileToDecompress.Extension.Length);
           using (FileStream decompressedFileStream = File.Create(newFileName))
           {
               using (GZipStream decompressionStream = new GZipStream(originalFileStream, CompressionMode.Decompress))
               {
                   decompressionStream.CopyTo(decompressedFileStream);
                   Console.WriteLine("Decompressed: {0}", fileToDecompress.Name);
               }
           }
       }
   }

Also I found a way to de serialize directly to object from https://www.codeart.dk/blog/2020/5/reading-very-large-gzipped-json-files-in-c/ But I didn't find a way to decompress and save to local drive

Upvotes: 0

Views: 701

Answers (0)

Related Questions