CODEWITHSUNDEEP
bashloopsawk
Schawn
Schawn

Reputation: 55

How to awk through multiple files?

I have hundreds of .dat file with data in side and I want to awk them with the command 

awk 'NR % 513 == 99' data.0003.127.dat > data.0003.127.Ma.dat

I tried to write a script file like

for i in {1 ... 9}; do
i=i*3
datafile = sprintf("data.%04d.127.dat",i)
outfile = sprintf("data.%04d.127.Ma.dat",i)
 awk 'NR % 513 == 99' datafile > outfile
done

I only need 0003 0006 0009 ... files, but the above script doesn't work fine. The error says

bash: ./Ma_awk.sh: line 3: syntax error near unexpected token `('
bash: ./Ma_awk.sh: line 3: `datafile = sprintf("data.%04d.127.dat",i)'

What shall I do next? I use ubuntu 14.04.

Upvotes: 0

Views: 79

Answers (2)

rici
rici

Reputation: 241671

In bash (since v4) you can write a sequence expression with an increment:

$ echo {3..27..3}
3 6 9 12 15 18 21 24 27

You can also include leading zeros, which will be preserved:

$ echo {0003..0027..3}
0003 0006 0009 0012 0015 0018 0021 0024 0027

So you could use the following:

for i in {0003..0027..3}; do
  awk 'NR % 513 == 99' "data.$i.127.dat" > "data.$i.127.Ma.dat"
done

Upvotes: 3

Pankrates
Pankrates

Reputation: 3094

There are multiple issues with your code, simply because they are bash syntax errors.

  1. Don't use spaces around variable assignment
  2. Brace expension looks like {1..9}
  3. Capturing stdout into a variable is done through =$() notation
  4. Reference variables with the dollar sign $
  5. Especially for filenames use double quotes " to ensure that the argument is considered as one

Now I have not considered the actual validity of your awk program but fixing the bash syntax errors would look something like the following

#!/usr/bin/env bash

for i in {1..9}; do
    datafile=$(printf "data.%04d.127.dat" $i)
    outfile=$(printf "data.%04d.127.Ma.dat" $i)
    awk 'NR % 513 == 99' "$datafile" > "$outfile"
done

This does not take care of the correct iteration bounds with an increment of three, but since you have not specified an upper bound I will leave that as an exercise to you

Upvotes: 2

Related Questions