CODEWITHSUNDEEP
bashfunctionshellgrep
john_black
john_black

Reputation: 227

getting the output of a grep command in a loop

I have a shell script that includes this search:

find . -type f -exec grep -iPho "barh(li|mar|ag)" {}  \;

I want to capture each string the grep command finds and send it a function I will create named "parser" 

parser(){ 
# do stuff with each single grep result found
}

how can that be done? is this right?

find . -type f -exec grep -iPho "barh(li|mar|ag)" {parser $1}  \;

I do not want to output the entire find command result to the function

Upvotes: 0

Views: 2152

Answers (3)

user1208081
user1208081

Reputation: 1117

i suggest you to use sed ,this is more powerful tool to do text processing. for example i want to add string "myparse" after the line that end as "ha",i can do this like 

# echo "haha" > text1
# echo "hehe" > text2
# echo "heha" > text3
# find . -type f -exec sed '/ha$/s/ha$/ha myparse/' {} \;
haha myparse
heha myparse
hehe

if you really want to replace the file ,not just print to stdout,you can do this like

# find . -type f -exec sed -i '/ha$/s/ha$/ha myparse/' {} \;

Upvotes: 0

PesaThe
PesaThe

Reputation: 7499

Only shell can execute a function. You need to use bash -c in your find in order to execute it. That is also the reason you need to export your function, so that the new process sees it.

parser() { 
    while IFS= read -r line; do
        echo "Processing line: $line"
    done <<< "$1"
}

export -f parser
find . -type f -exec bash -c 'parser "$(grep -iPho "barh(li|mar|ag)" "$1")"' -- {} \;

The code above will send all occurrences from file1, then file2 etc to your function to process. It will not send each line one by one and therefore you need to loop over the lines in your function. If there is no occurrence of your regex in a file, it will still call your function with an empty input!

That might not be the best solution for you so let's try to add the loop inside the bash -c statement and really process the lines one by one:

parser() { 
    echo "Processing line: $1"
}

export -f parser
find . -type f -exec bash -c 'grep -iPho "barh(li|mar|ag)" "$@" | while IFS= read -r line; do parser "$line"; done' -- {} +

EDIT: Very nice and simple solution not using bash -c suggested by @gniourf_gniourf:

parser() { 
   echo "Processing line: $1"
}

find . -type f -exec grep -iPho "barh(li|mar|ag)" {} + | while IFS= read -r line; do parser "$line"; done

This approach works fine and it will process each line one by one. You also do not need to export your function with this approach. But you have to care for some things that might surprise you.

Each command in a pipeline is executed in its own subshell, and any variable assignment in your parser function or your while in general will be lost after returning from that very subshell. If you are writing a script, simple shopt -s lastpipe will suffice and run the last pipe command in the current shell environment. Or you can use process substitution: 

parser() {
   echo "Processing line: $1"
}

while IFS= read -r line; do 
   parser "$line"; 
done < <(find . -type f -exec grep -iPho "barh(li|mar|ag)" {} +)

Note that in the previous bash -c examples, you will experience the same behavior and your variable assignments will be lost as well.

Upvotes: 1

Martin Povolny
Martin Povolny

Reputation: 111

You need to export your function.

You also need to call bash to execute the function.

parser() {
  echo "GOT: $1"
}
export -f parser

find Projects/ -type f -name '*rb' -exec bash -c 'parser "$0"' {} \;

Upvotes: 0

Related Questions