CODEWITHSUNDEEP
bash
mkalkov
mkalkov

Reputation: 1494

Matching one line from continuous stream in Linux shell

How can I make the following commands exit immediately after the first line is matched? I understand that SIGPIPE is not sent to cat until it tries to write next time (tail bug report), but I don't understand how to solve this issue.

cat <( echo -ne "asdf1\nzxcv1\nasdf2\n"; sleep 5; echo -ne "zxcv2\nasdf3\n" ) | grep --line-buffered zxcv | head --lines=1
cat <( echo -ne "asdf1\nzxcv1\nasdf2\n"; sleep 5; echo -ne "zxcv2\nasdf3\n" ) | grep --max-count=1 zxcv

NB: I actually had tail --follow before the pipesign, but replaced it with catand sleep to simplify testing. The shell in question is GNU bash 4.4.12(1)-release, and I'm running MINGW that came with Git-for-Windows 2.12.2.2.

CLARIFICATION: I have a jboss server which is started in a docker container and which outputs couple thousand lines of text within three minutes to a log file. My goal is to watch this file until a status line is printed, analyze line contents and return it to a human or Jenkins user. Of course, I can grep whole file and sleep for a second in a loop, but I'd rather avoid this if at all possible. Furthermore, this looping would interfere with my usage of timeout routine to limit maximum execution time. So, is it possible to listen for a pipe until a certain line appears and stop as soon as that happens?

Related question: Why does bash ignore SIGINT if its currently running child handles it?

Upvotes: 2

Views: 284

Answers (2)

mkalkov
mkalkov

Reputation: 1494

I've ended up doing following to be able to break following log both on a matching line and after a timeout.

#!/bin/sh

TOP_PID=$$
container_id="$1"
LOG_PATH=/opt/jboss/jboss-eap-6.2/standalone/log/server.log

await_startup () {
  status=$(check_status)
  follow_log --timeout $timeout &
  local bgjob_pid; local bgjob_status;
  bgjob_pid=$(jobs -p)
  test -n "$bgjob_pid" || die "Could not start background job to follow log."
  bgjob_status=true
  while [ "$status" = "not started" ] && $bgjob_status; do
    sleep 1s
    status=$(check_status)
    if kill -0 $bgjob_pid 2>/dev/null; then
      bgjob_status=true
    else
      bgjob_status=false
    fi
  done
  kill -KILL $bgjob_pid 2>/dev/null
}

follow_log () {
  # argument parsing skipped...
  docker exec $container_id timeout $timeout tail --follow=name ---disable-inotify --max-unchanged-stats=2 /$LOG_PATH
}

check_status () {
  local line;
  line=$(docker exec $container_id grep --extended-regexp --only-matching 'JBoss EAP .+ started.+in' /$LOG_PATH | tail --lines=1)
  if [ -z "$line" ]; then
    printf "not started"
  elif printf "%s" "$line" | grep --quiet "with errors"; then
    printf "started and unhealthy"
  else
    printf "healthy"
  fi
}

die () {
  test -n "$1" && printf "%s\n" "$1"
  kill -s TERM $TOP_PID
  return 1
} 1>&2

Upvotes: 0

l0b0
l0b0

Reputation: 58988

Interesting question! I've verified that head dies after printing the first line (removed background job noise):

$ (printf '%s\n' a b a; sleep 5; printf '%s\n' a) | grep --line-buffered a | head --lines=1 & sleep 1; pstree $$
a
bash─┬─bash───sleep
     ├─grep
     └─pstree

At first glance, it appears head doesn't send SIGPIPE, but I get conflicting information from running strace grep:

$ (printf '%s\n' a b a; sleep 10; printf '%s\n' a) | strace grep --line-buffered a | head --lines=1
…
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=21950, si_uid=1000} ---
+++ killed by SIGPIPE +++

… and killing grep:

$ (printf '%s\n' a b a; sleep 10; printf '%s\n' a) | grep --line-buffered a | head --lines=1 & sleep 1; kill -PIPE $(pgrep grep); sleep 5; pstree $$
a
bash─┬─bash───sleep
     └─pstree

Killing grep and then sleep fixes the issue:

$ (printf '%s\n' a b a; sleep 10; printf '%s\n' a) | grep --line-buffered a | head --lines=1 & sleep 1; kill -PIPE $(pgrep grep); sleep 1; kill -PIPE $(pgrep sleep); sleep 5; pstree $$
a
bash───pstree

Conclusion: WTF?

Upvotes: 1

Related Questions