CODEWITHSUNDEEP
linuxbashloops
lightsnail
lightsnail

Reputation: 788

Loop every three consecutive rows in linux

I have a file hundred.txt containing 100 rows.

For example:

1 0 0 1
1 1 0 1
1 0 1 0
1 0 1 0
0 1 1 0
....
1 0 0 1

I need to manipulate some calculations within every 3 consecutive rows, for instance, I need to use the Row1-Row3 first to do my calculation:

1 0 0 1
1 1 0 1
1 0 1 0

then the Row2-Row4:

1 1 0 1
1 0 1 0
1 0 1 0

...... the Row98-Row100.

Each output will generate a file (e.g. Row1.txt, Row2.txt,... Row98.txt), How can I solve this problem? Thank you.

Upvotes: 2

Views: 365

Answers (3)

karakfa
karakfa

Reputation: 67507

awk to the rescue!

$ awk 'NR>2{printf "%s", a2 ORS a1 ORS $0 ORS > FILENAME"."(++c)}
           {a2=a1;a1=$0}' file

for the input file

$ cat file
1 0 0 1
1 1 0 1
1 0 1 0
1 0 1 0
0 1 1 0

generates these 3

$ head file.{1..3}
==> file.1 <==
1 0 0 1
1 1 0 1
1 0 1 0

==> file.2 <==
1 1 0 1
1 0 1 0
1 0 1 0

==> file.3 <==
1 0 1 0
1 0 1 0
0 1 1 0

you can embed your computation is the script and output only the results but you didn't provide any details on that.

Explanation

NR>2 starting third row
printf ... start printing last 3 rows
> FILENAME"."(++c) to a file derived from input filename with counter suffix

a2=a1;a1=$0 update last two rows

if your rolling window is small n you can scale this script by changing NR>(n-1) and keeping track of last rows in a(n-1)...a1 and printing accordingly. If n is large, better to use a array (or better a circular array).

This is perhaps the most generic version...

$ awk -v n=3 'NR>n-1{fn=FILENAME"."c; 
                     for(i=c+1;i<c+n;i++) printf "%s\n", a[(i-n)%n] > fn;
                     print > fn} 
                    {a[(c++)%n]=$0}' file

Upvotes: 2

kojiro
kojiro

Reputation: 77137

One hundred rows of four binary-valued columns is not too much; just read it all in at once.

mapfile -t rows < inputfile
for r in "${!rows[@]}"; do # loop by row index
    (( r >= 2 )) || continue
    # process "${rows[r-2]}" "${rows[r-1]}" and "${rows[r]}"
    # into file Row$((r-1))
done

If the quantity of data grows significantly, you really want to use a better tool, such as Python+numpy (because your data looks like binary matrices).

Upvotes: 2

chepner
chepner

Reputation: 531898

bash isn't a great choice for data processing tasks, but it is possible (albeit slow):

{ read row1
  read row2
  count=0
  while read row3; do
    # Do something with rows 1-3
    { echo $row1 $row2 $row3; } > Row$((count+=1)).txt
    # Slide the window
    row1=$row2
    row2=$row3
  done
} < hundred.txt

Upvotes: 3

Related Questions