CODEWITHSUNDEEP
bashawk
jeremysprofile
jeremysprofile

Reputation: 11445

How to redirect to a file in multiple way in awk

I have a CSV. I am trying to write the header and an arbitrary percentage of lines to one file, and the header and the rest of the lines to another file.

awk '
  BEGIN{srand()}
  FNR==1 {print | "tee f1 f2"; close("f1"); close("f2")}
  FNR!=1 {if (rand() <= 0.2) print >> "f1"; else print >> "f2"}
' my.csv

I am using gawk. I read the redirection man page. I thought that adding close("f1"); close("f2"); to my first awk action would help, but it doesn't appear to have done anything (the behavior is the same without them).

When I rewrite it to not use tee, it does work:

awk '
  BEGIN{srand()}
  FNR==1 {print > "f1"; print > "f2"}
  FNR!=1 {if (rand() <= 0.2) print >> "f1"; else print >> "f2"}
' my.csv

Question: How do I properly write to a file I've already written to with tee?

Upvotes: 0

Views: 99

Answers (2)

Diego Torres Milano
Diego Torres Milano

Reputation: 69218

The second option, without tee, is preferred as it does not create an unnecessary process.

You can simplify it a bit more.

awk '
  BEGIN{srand()}
  FNR==1 {print > "f1"; print > "f2"}
  FNR!=1 {print > ("f" (rand() <= 0.2 ? "1" : "2"))}
' my.csv

Notice, you don't have to use >> in awk as you do in bash.

Upvotes: 0

tshiono
tshiono

Reputation: 22022

Please try to modify the FNR==1 line into something like:

FNR==1 {print | "tee f1 f2"; close("f1"); close("f2"); system("sleep 60")}

then execute the script and invoke ps aux | grep tee from another terminal. You'll see the tee process is still working. It keeps on blocking "f1" and "f2" from the redirection within the awk script.
Please try instead:

FNR==1 {print | "tee f1 f2"; close("tee f1 f2")}

which safely closes the tee subprocess and enables the following redirections.

Upvotes: 2

Related Questions