CODEWITHSUNDEEP
linuxbashshellgreptail
Francisco Noriega
Francisco Noriega

Reputation: 14584

How to execute a command with each GREP result while TAILING a file

TL;DR: How to execute a command on each grep match that is produced while piping tail -f into grep.

I am currently using tail -f file | grep exceptionto follow all the exceptions thrown by a service. One of the issues is that the line that has the exception doesn't have all the information I need, but it does contain an ID that identifies all the log lines generated by that request.

What I want to do is to to print all lines that have a certain ID once grep matches a line with exception.

Using the help from this question How to grep and execute a command (for every match)

I manage to make it work for the CAT command, like this:

cat myFile | 
egrep -i "exception" | #find lines with exception
egrep -o "ID=[A-Z]{10}" | #for those lines, select out only the id
while read line;  do #for each id
   cat myFile | grep $line; #show all the lines that have that id
done

that works fine and prints all the lines with matching ID, however when I change the cat to tail -f it wont work, it wont print anything. What am I doing wrong?

Upvotes: 3

Views: 2190

Answers (1)

ghoti
ghoti

Reputation: 46856

The problem you're having is likely that grep is buffering its output when it sees that its output is another pipe. Your command might eventually produce output if you wait long enough for a buffer to fill up.

Instead, try the following:

< myFile egrep --line-buffered -i "exception" \
| egrep --line-buffered -o "ID=[A-Z]{10}" \
| while read line; do
    cat myFile | grep "$line"
  done

(Yes, that input redirection option should work just fine. :] )

The relevant man page excerpt is:

   --line-buffered
          Use line buffering on output.  This can cause a performance penalty.

which obviously wouldn't have helped you much unless you knew what you were looking for. :-P

Note that in the Linux world, you're using Linux, where your man page probably states that "Direct invocation as either egrep or fgrep is deprecated, but is provided to allow historical applications that rely on them to run unmodified." Though to be honest, I don't see egrep ever disappearing, and other platforms (FreeBSD, OSX) make no mention of deprecation.

Upvotes: 3

Related Questions