CODEWITHSUNDEEP
bash
4v1
4v1

Reputation: 11

Bash Iterative approach in place of process substitution not working as expected

complete bash noob here. Had the following command (1.) and it worked as expected but it seemed a bit naive for what I needed:

  1. Essentially generating a wordlist from a messy input file with tab delimiters
cat users.txt | tee >(cut -f 1 >> cut_out.txt) >(cut -f 2 >> cut_out.txt) >(cut -f 3 >> cut_out.txt)  >(cut -f 4 >> cut_out.txt)

Output:

W Humphrey  
SummersW  
FoxxR  
noreply  
DaibaN  
PeanutbutterM  
PetersJ  
DaviesJ  
BlaireJ  
GongoH  
MurphyF  
JeffersD  
HorsemanB  
...
  1. Thought I could cut down on the ridiculous command above with the following
cat users.txt | for i in {1..4}; do cut -f $i >> cut_out.txt; done

Output:

HumphreyW

The command above only returned a single word from the list and some white-space.

  1. The solution. I knew that I could get it working logically by simply looping the entire command instead, this did exactly what I wanted but just wanted to know why the command above (2.) returned an almost empty file?
for i in {1..4}; do cat users.txt | cut -f $i >> cut_out.txt; done

Have a solution, more-so wanted an explanation because I am dumb and still learning about I/O in bash. Cheers.

Upvotes: 1

Views: 47

Answers (2)

tripleee
tripleee

Reputation: 189487

If the order of the output is unimportant, and there are only four coumns in the file, simply

tr '\t' '\n' <users.txt >cut_out.txt

If you only want the first four columns in any order,

cut -f1-4 users.txt |
rt '\t' '\n' >cut_out.txt

(Thanks to @KamilCuk for raising this in a comment.)

Otherwise your third attempt is basically fine, though you want to avoid the useless cat and redirect only once;

for i in {1..4}; do
    cut -f "$i" users.txt
done > cut_out.txt

This is obviously less efficient than only reading the file once. If the file is small enough to fit into memory, you could write a simple Awk script to read it once and split it up into variables, and then write out these variables in the order you want.

The second attempt is wrong because cat only supplies a single instance of the data to the pipe, and the first iteration of the loop consumes it all.

Upvotes: 1

Fravadona
Fravadona

Reputation: 17075

Just a remark

awk -F '[\t]' '{for(i = 1; i <= 4; i++) print $i}' users.txt > cut_out.txt

Is basically what your cat ... | tee >(cut ...) ... does.

Upvotes: 1

Related Questions