CODEWITHSUNDEEP
linuxunixgzipsolarissunos
AKIWEB
AKIWEB

Reputation: 19612

Combine files in one

Currently I am in this directory-

/data/real/test

When I do ls -lt at the command prompt. I get like below something-

REALTIME_235000.dat.gz
REALTIME_234800.dat.gz
REALTIME_234600.dat.gz
REALTIME_234400.dat.gz
REALTIME_234200.dat.gz

How can I consolidate the above five dat.gz files into one dat.gz file in Unix without any data loss. I am new to Unix and I am not sure on this. Can anyone help me on this?

Update:-

I am not sure which is the best way whether I should unzip each of the five file then combine into one? Or combine all those five dat.gz into one dat.gz?

Upvotes: 6

Views: 11508

Answers (3)

Alexander Kosenkov
Alexander Kosenkov

Reputation: 1595

It seems almost like a black magic, but you can actually concatenate GZ files directly!

The format was made for this (along with MP3). Internally, GZ is organized in independent chunks of compressed stream, each with its own header, compression dictionary, checksum, and so on.

So when you concatenate several GZ files, the uncompressed stream is exactly the concatenation of original files.

Upvotes: -1

Mark Adler
Mark Adler

Reputation: 112219

What do you want to happen when you gunzip the result? If you want the five files to reappear, then you need to use something other than the gzip (.gz) format. You would need to either use tar (.tar.gz) or zip (.zip).

If you want the result of the gunzip to be the concatenation of the gunzip of the original files, then you can simply cat (not zcat or gzcat) the files together. gunzip will then decompress them to a single file.

cat [files in whatever order you like] > combined.gz

Then:

gunzip combined.gz

will produce an output that is the concatenation of the gunzip of the original files.

The suggestion to decompress them all and then recompress them as one stream is completely unnecessary.

Upvotes: 5

Ivan Nevostruev
Ivan Nevostruev

Reputation: 28713

If it's OK to concatenate files content in random order, then following command will do the trick:

zcat REALTIME*.dat.gz | gzip > out.dat.gz

Update

This should solve order problem:

zcat $(ls -t REALTIME*.dat.gz) | gzip > out.dat.gz

Upvotes: 12

Related Questions