CODEWITHSUNDEEP
linuxawk
Nikhilesh Sarish Kulkarni
Nikhilesh Sarish Kulkarni

Reputation: 81

Running find, grep and awk script

I am using find to search for file. then I pass this file as argument for grep. I want to run awk script on each result? How can I do this ?

so lets say, find returns 5 files, grep on each file return 15 lines and on these 15 lines, i want to run awk script.

I tried following but it error out

find . -type f -name test.log | xargs grep -A 15 "Data starts now" | xargs awk -f postprocess.awk

can anybody suggest anything wrong with syntax ?

Lets say file test1.log is 

num1 104
num2 434
Num3 572
Data starts now
num1 04
num2 34
Num3 72

Lets say file test2.log is 

num1 203
num2 135
Num3 098
Data starts now
num1 17
num2 33
Num3 89

Lets say file test3.log is 

num1 924
num2 834
Num3 532
Data starts now
num1 34
num2 63
Num3 89

postprocess.awk is

{
if($1=="num1")
   {
   num1_value =$2;
   }

if($1=="num2")
   {
   num2_value =$2;
   }


if($1=="num3")
   {
   num3_value =$2;
   }



}

END {
mult=num1_value*num2_value;
print "Multiplication is " mult;
}

if I run

find . -type f -name "test*.log" -exec grep -A15 "Data starts now" {} + | awk -f postprocess.awk

I should have got 3 output but get only 1 line with wrong result

Multiplication is 0

Upvotes: 1

Views: 198

Answers (3)

SiegeX
SiegeX

Reputation: 140437

If you have GNU awk you can do everything in one line:

$ awk 'c{c--;a[$1]=$2} /Data starts now/{c=15} \
    ENDFILE{m=1;c=0;for(i in a){m*=a[i]}print FILENAME": Multiplication is "m}' test*.log
test1.log: Multiplication is 9792
test2.log: Multiplication is 49929
test3.log: Multiplication is 190638

Explanation

awk '
  c{                       # if c is non-zero (c is 0 when script is 1st ran)
     c--                   # decrement c
     a[$1]=$2              # Create a hash-map with $1 as the key and $2 as the value
   }
  /Data starts now/{c=15}  # When regex matches, set c to 15
  ENDFILE{                 # True when the last record of a file has been read (gawk only)
    m=1                    # Set m to one so multiplication doesnt return 0
    c=0                    # Set c to 0 in case file has less than 15 lines after match
    for(i in a){           # For each key/value pair...
      m*=a[i]              # Multiply them together and store result in m
    }
    print FILENAME": Multiplication is "m}
' test*.log

Upvotes: 1

alani
alani

Reputation: 13079

It will be clearer if you create another helper script (postprocess.sh)

for f in $*
do
   grep -A 15 "Data starts now" $f | awk -f postprocess.awk
done

and then run this inside the -exec:

find . -type f -name "test*.log" -exec sh postprocess.sh {} +

Upvotes: 1

Freddy
Freddy

Reputation: 4708

To run grep and awk on each individual file found by find, you can use the -exec action and start a small shell script:

find . -type f -name "test*.log" -exec sh -c '
  for file; do
    grep -A15 "Data starts now" "$file" | awk -f postprocess.awk
  done
' sh {} +

Upvotes: 1

Related Questions