CODEWITHSUNDEEP
fileunixconcatenation
JDS
JDS

Reputation: 16988

Unix combine a bunch of text files without taking extra disk space?

I have a bunch of text files I need to temporarily concatenate so that I can pass a single file (representing all of them) to some post-processing script.

Currently I am doing:

zcat *.rpt.gz > tempbigfile.txt

However this tempbigfile.txt is 3.3GB, while the original size of the folder with all the *.rpt.gz files is only 646MB! So I'm temporarily quadroupling the disk space used. Of course after I can call myscript.pl with tempbigfile.txt, it's done and I can rm the tempbigfile.txt.

Is there a solution to not create such a huge file and still get all those files together in one file object?

Upvotes: 0

Views: 148

Answers (1)

Blender
Blender

Reputation: 298206

You're deflating the files with zcat, so you should compress the text once more with gzip:

zcat *.rpt.gz | gzip > tempbigfile.txt

Upvotes: 2

Related Questions