CODEWITHSUNDEEP
bashloopswildcardpastevariable-expansion
Stonecraft
Stonecraft

Reputation: 814

bash paste: loop through pairs of files based on wildcard, generate separate output files

I am trying to get the paste command to loop through pairs of files, pasting them together and outputing each as a unique file. I've tried a lot of things, here are a few:

 for i in *_temp4.csv; do paste *_temp4.csv *_temp44.csv > ${i}_out.csv; done
 #Each output contains each input file (rather than pairs). Obviously this is because of the * wildcard

 for i in *_temp2.csv_temp4.csv; do paste $_temp2_temp4.csv $_temp3_temp44.csv > ${i}_out.csv; done

no error, empty output files

 for i in *_temp2.csv_temp4.csv; do paste ${_temp2_temp4.csv} ${_temp3_temp44.csv} > ${i}_out.csv; done

output:

 combo15.awk: line 12: ${_temp2_temp4.csv}: bad substitution

I think I must be missing something very basic about how $ gets used, but I've been googling all night to no avail.

my entire code, for context, although I don't see why the previous lines should influence anything about this.

 for i in *.dat; do awk 'NR > 23 { print }' ${i} > ${i}_temp1.csv; done

 for i in *_temp1.csv; do awk 'BEGIN{OFS=FS=","}$2==0{$2="between"}BEGIN{OFS=FS=","}$2==1{$2="lego"}BEGIN{OFS=FS=","}$2==2{$2="pin"}BEGIN{OFS=FS=","}$2==3{$2="dice"}BEGIN{OFS=FS=","}$2==4{$2="jack"}BEGIN{OFS=FS=","}$2==8{$2="escape"}{print}'  ${i} > ${i}_temp2.csv; done

 for i in *_temp2.csv; do awk -v OFS="," '{$4 = $1 - prev1; prev1 = $1; print;}' ${i} > ${i}_temp3.csv; done  

 for i in *_temp2.csv; do awk -F "," 'BEGIN{print "new line"}{print $2}' ${i} > ${i}_temp4.csv; done

 for i in *_temp3.csv; do awk -F "," '{print $5}' ${i} > ${i}_temp44.csv; done

 for i in *_temp2.csv_temp4.csv; do paste $_temp2_temp4.csv    $_temp3_temp44.csv > ${i}_out.csv; done

Upvotes: 0

Views: 391

Answers (1)

Sleafar
Sleafar

Reputation: 1526

Your problem is, that names of your files grow uncontrollably. This change should solve this problem:

for i in *.dat; do awk 'NR > 23 { print }' ${i} > ${i}_temp1.csv; done

for i in *.dat; do awk 'BEGIN{OFS=FS=","}$2==0{$2="between"}BEGIN{OFS=FS=","}$2==1{$2="lego"}BEGIN{OFS=FS=","}$2==2{$2="pin"}BEGIN{OFS=FS=","}$2==3{$2="dice"}BEGIN{OFS=FS=","}$2==4{$2="jack"}BEGIN{OFS=FS=","}$2==8{$2="escape"}{print}'  ${i}_temp1.csv > ${i}_temp2.csv; done

for i in *.dat; do awk -v OFS="," '{$4 = $1 - prev1; prev1 = $1; print;}' ${i}_temp2.csv > ${i}_temp3.csv; done

for i in *.dat; do awk -F "," 'BEGIN{print "new line"}{print $2}' ${i}_temp2.csv > ${i}_temp4.csv; done

for i in *.dat; do awk -F "," '{print $5}' ${i}_temp3.csv > ${i}_temp44.csv; done

for i in *.dat; do paste ${i}_temp4.csv    ${i}_temp44.csv > ${i}_out.csv; done

Upvotes: 1

Related Questions