CODEWITHSUNDEEP
pythoncompression7ziplzma
Cryptite
Cryptite

Reputation: 1466

Compress a folder recursively as 7z with PyLZMA and py7zlib

Through much trial and error, I've figured out how to make lzma compressed files through PyLZMA, but I'd like to replicate the seemingly simple task of compressing a folder and all of its files/directories recursively to a 7z file. I would just do it through 7z.exe, but I can't seem to be able to catch the stdout of the process until it's finished and I would like some per-7z-file progress as I will be compressing folders that range from hundreds of gigs to over a terabyte in size. Unfortunately I can't provide any code that I have tried, simply because the only thing I've seen examples of using py7zlib are extracting files from pre-existing files. Has anybody had any luck with the combination of these two or could provide some help?

For what it's worth, this would be on Windows using python 2.7. Bonus points if there some magic multi-threading that could occur here, especially given how slow lzma compression seems to be (time, however, is not an issue here). Thanks in advance!

Upvotes: 1

Views: 5238

Answers (1)

Pedro Romano
Pedro Romano

Reputation: 11203

A pure Python alternative is to create .tar.xz files with a combination of the standard library tarfile module and the liblzma wrapper module pyliblzma. This will create files comparable to in size to .7z archives:

import tarfile
import lzma

TAR_XZ_FILENAME = 'archive.tar.xz'
DIRECTORY_NAME = '/usr/share/doc/'

xz_file = lzma.LZMAFile(TAR_XZ_FILENAME, mode='w')

with tarfile.open(mode='w', fileobj=xz_file) as tar_xz_file:
    tar_xz_file.add(DIRECTORY_NAME)

xz_file.close()

The tricky part is the progress report. The example above uses the recursive mode for directories of the tarfile.TarFile class, so there the add method would not return until the whole directory was added.

The following questions discuss possible strategies for monitoring the progress of the tar file creation:

Upvotes: 2

Related Questions