CODEWITHSUNDEEP
bashunixawk
ubi
ubi

Reputation: 4399

Capture logs for a duration

I'm trying to capture logs for a set period, parse it and produce a report. Here's what I do 

(tail -F log_file | grep --line-buffered some_text | awk '{process lines} END {produce report}') & pid=$! && disown
sleep 60 
pkill -TERM -P $pid
kill -TERM $pid

Explanation:

  1. tail a log file and pipe to grep, awk to process and produce report at END. Run these in a command group (within ())
  2. Wait 60 seconds
  3. kill the children of the process group (tail, grep, awk)
  4. kill the command group

Now the problem is when awk is killed it won't write the report (complete the END part)! What am I doing wrong here? Can you suggest a workaround?

Upvotes: 1

Views: 28

Answers (1)

tripleee
tripleee

Reputation: 189377

You already explained your problem. "When Awk is killed, it won't write the report."

The proper solution is to only kill tail and then wait for the rest of the pipeline to finish.

If your tail supports the --pid= argument, that's easy -- just start a sentinel sleep process in the background, run for as long as you need to, kill the sentinel, wait for tail to exit. Or use the sleep 60 you are already using; just start it before the tail pipeline instead.

sleep 60 &
zzz=$!
tail -F log_file --pid=$zzz | awk '/some_text/ {process lines} END {produce report}'
wait $zzz

(Note also the refactoring to lose the useless grep | awk.)

If it doesn't, things easily get a lot more involved. I can imagine that it might be possible by setting a clever trap in the subshell, but I would probably simply write a short Python or Perl wrapper with a signal handler to write the report when terminated.

Upvotes: 2

Related Questions