CODEWITHSUNDEEP
bashawksplit
sdgaw erzswer
sdgaw erzswer

Reputation: 2382

Bash splitting a large gzipped file

I've been recently dealing with some file processing, and I am trying to write a bash one-liner, which would look something like:

zcat largefile.gz | split_by_delimiter_into_separate_files

Things I tried:

zcat largefile.gz | awk '{print $0 " //"> "separate_file" NR}' RS='//'

The delimiter I am trying to split upon is "//". I know something like python could probably solve this into a couple of lines, yet my project is not python-based and this is thus not an option..

Upvotes: 1

Views: 47

Answers (2)

damienfrancois
damienfrancois

Reputation: 59140

You can use split which I believe does exactly what you need: 

zcat largefile.gz | split -p '//' - separate_file_

will create files prefixed with separate_file_ with the content of large file, split on //

Upvotes: 0

komar
komar

Reputation: 881

Try it like this:

zcat largefile.gz | awk -vRS='//' '{print $0 " //"> "separate_file" NR}'

Upvotes: 2

Related Questions