CODEWITHSUNDEEP
linuxbashcsvcat
Konstantin Popov
Konstantin Popov

Reputation: 1736

Split many CSV files into a few bigger files in Linux

I have a bunch of small CSV files (a few hundred files about 100 MB each) that I want to pack into several bigger files. I know how to join all (or a subset) of those files into one file - I simply need to use cat command in Linux and redirect its output to a file. My problem is the result files have to be not bigger than some size (say, 5 GB), i.e. merging all small files into one is not a solution because the resulting file will be too big. So, I am wondering if there is a way to do it in the command line that would be simpler than writing a bash script looping over the directory?

Thanks.

Upvotes: 0

Views: 61

Answers (1)

Mr. Llama
Mr. Llama

Reputation: 20889

The split command does exactly what you need. You can have it split STDIN to different outputs based on size or number of lines. You can also specify the output file suffix.

Upvotes: 2

Related Questions