CODEWITHSUNDEEP
c#7zip
Paul S.
Paul S.

Reputation: 66384

Creating solid archive with internal path structure from generated files

I'm trying to generate a solid archive from hundreds of files which I will later compress, but everything I attempt seems to hit a problem.

The files to be archived are being generated in my program and I access them as Streams. They need to be organised in paths up to any length.

Currently I'm passing them into the stdin of 7z.exe, I'm achieving the command (notice -mx0 because I don't want to compress on this step)

7z.exe a -t7z "file_out_path" -mx0 -si"internal_archive_path" < stream

using the following code 

public static void Run(string exe, string args = null, System.IO.Stream stdin = null)
{
    Process p = new Process();
    p.StartInfo.UseShellExecute = false;
    if (stdin != null)
        p.StartInfo.RedirectStandardInput = true;
    p.StartInfo.FileName = exe;
    if (args != null)
        p.StartInfo.Arguments = args;
    p.StartInfo.CreateNoWindow = true;
    p.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
    p.Start();
    if (stdin != null)
    {
        // stdin.CopyTo(p.StandardInput.BaseStream);
        int readLen;
        int buffSize = 4096;
        byte[] buff = new byte[buffSize];
        while (0 < (readLen = stdin.Read(buff, 0, buffSize)))
        {  
            p.StandardInput.BaseStream.Write(buff, 0, readLen);
            p.StandardInput.BaseStream.Flush();
        }
        p.StandardInput.Close();
    }
    p.WaitForExit();
}

// for loop
    // make System.IO.Stream stream, etc
    Run(bin7z, "a -t7z \"" + solid_archive_path + "\" -mx0 -si\"" + internal_archive_path + "\"", stream);

When it starts it seems to be working fine and generates the desired path structure, however as the archive starts to add up in size it becomes obvious that every time I'm adding a new file it's copying the entire archive to a .tmp. This is unacceptable because the archive is expected to grow to ~3 GB and contain several hundreds files meaning the current behaviour results in hundreds of GB being copied over and over again in total and the copying takes much longer than any other step.

How can I prevent this behaviour? I'm open to other archiving solutions.

Upvotes: 0

Views: 146

Answers (1)

Graffito
Graffito

Reputation: 1718

Instead of running 7z to create an uncompressed 7zip file, you may copy the files to a new temporary directory with the desired path structure.

Once all files copied, run a 7z command to create the compressed/uncompressed 7z archive from the temp directory and finally clear the temp directory.

Upvotes: 0

Related Questions