CODEWITHSUNDEEP
linuxbashshellwhile-looppipe
johnathon doeth
johnathon doeth

Reputation: 15

Bash - Piping output of command into while loop

I'm writing a Bash script where I need to look through the output of a command and do certain actions based on that output. For clarity, this command will output a few million lines of text and it may take roughly an hour or so to do so.

Currently, I'm executing the command and piping it into a while loop that reads a line at a time then looks for certain criteria. If that criterion exists, then update a .dat file and reprint the screen. Below is a snippet of the script.

eval "$command"| while read line ; do
    if grep -Fq "Specific :: Criterion"; then
        #pull the sixth word from the line which will have the data I need
        temp=$(echo "$line" | awk '{ printf $6 }')
        #sanity check the data
        echo "\$line = $line"
        echo "\$temp = $temp"

        #then push $temp through a case statement that does what I need it to do.
    fi
done

So here's the problem, the sanity check on the data is showing weird results. It is printing lines that don't contain the grep criteria.

To make sure that my grep statement is working properly, I grep the log file that contains a record of the text that is output by the command and it outputs only the lines that contain the specified criteria.

I'm still fairly new to Bash so I'm not sure what's going on. Could it be that the command is force feeding the while loop a new $line before it can process the $line that met the grep criteria?

Any ideas would be much appreciated!

Upvotes: 1

Views: 3544

Answers (2)

gbtimmon
gbtimmon

Reputation: 4342

How does grep know what line looks like? 

if ( printf '%s\n' "$line" | grep -Fq "Specific :: Criterion"); then

But I cant help feel like you are overcomplicating a lot. 

function process() { 
    echo "I can do anything I want"
    echo " per element $1"
    echo " that I want here"
}

export -f process

$command | grep -F "Specific :: Criterion" | awk '{print $6}' | xargs -I % -n 1 bash -c "process %";

Run the command, filter only matching lines, and pull the sixth element. Then if you need to run an arbitrary code on it, send it to a function (you export to make it visible in subprocesses) via xargs.

Upvotes: 3

Abis
Abis

Reputation: 165

What are you applying the grep on ?

Modify

if grep -Fq "Specific :: Criterion"; then

as below

if ( echo $line | grep -Fq "Specific :: Criterion" ); then

Upvotes: 1

Related Questions