CODEWITHSUNDEEP
linuxbashpipeconditional-statements
marco
marco

Reputation: 1752

bash, insert a condition in a command with pipes

I need to create a file with some fields of another very huge file but only if field 3 is empty.

Actually I'm doing this:

cut -f 1,3,4,6 allCountries.txt | sort -u > cities_nostate.txt

then, with another script, I go through all the lines of the cities_nostate.txt file and check all the fields but this takes hours.

Is it possible to put a condition in the command in order to fast filter and remove all the useless lines (those with field 3 not empty) ?

After having accepted the answer, this is how I solved it:

awk -F'\t' '$3 == "" && $1 != "" && $4 != "" && $6 != "" {print $1, "\t", $4, "\t", $6}' allCountries.txt | sort -u >

Field 3 must be empty but all the others not. I need a tab separator between the printed fields as there might be spaces inside each field.

Upvotes: 1

Views: 162

Answers (2)

Tom Fenech
Tom Fenech

Reputation: 74595

If your fields are separated by tab characters and you only want to print the lines for which field 3 is empty, you can use this:

awk -F'\t' '$3 == ""{print $1,$3,$4,$6}' allCountries.txt

In awk, the default input field separator is any number of space characters ([[:space:]]+), so you cannot detect an empty field without changing it to something else.

To print the output columns separated by tabs, you can set the Output Field Separator variable OFS. This can be done in the BEGIN block before the file is processed:

awk 'BEGIN{FS=OFS="\t"}$3 == ""{print $1,$3,$4,$6}' allCountries.txt

I've assigned the input Field Separator to a tab character at the same time.

Upvotes: 2

anubhava
anubhava

Reputation: 785068

Instead of cut you can use awk:

awk '$3 == "" {print $1,$3,$4,$6}' allCountries.txt | sort -u > cities_nostate.txt

$3 == "" will print fields only when field 3 is empty.

Upvotes: 2

Related Questions