CODEWITHSUNDEEP
linuxbashloggingmonitoringtail
Jack C.
Jack C.

Reputation: 21

Prepend tail -f output with filename

To monitor a system I need to constantly have a log tail running (it greps out some data as well). It's basically

tail -f *.log' | egrep -v '(str1|str2)

The output of tail is very noisy and it constantly prints the file that it is currently tailing from.

==> example1.log <==
log example 1
==> example2.log <==
log example 2

I would like to be able to change the output I'm getting from tail -f *.log and prefix the output with the filename on a per line basis so that the output looks like this:


example1.log:log example 1
example2.log:log example 2

I'm looking for a simple hack to achieve this as it's simply for my monitoring of log files in real time using tail -f. It might be possible to even replace the output using sed to take away the newlines and other data and prefix.

Anyone have a clue? Would be greatly appreciated for this monitoring hack.

Thanks! JC

I've tried replacing the output using sed but it's not logical.

Because I'm grepping data out, the output of tail is constantly repeating lines such as

==> example1.log <==
some data
==> example2.log <==
==> example1.log <==
==> example2.log <==
some data
==> example1.log <==

It's quickly become a kind of difficult task and I really just need the filename prepended to the output as it were with grep.

I need to monitor the realtime logs and I haven't found multitail or other commands to meet the needs, if someone has a working example I'd be forever grateful.

Upvotes: 2

Views: 473

Answers (1)

William Pursell
William Pursell

Reputation: 212288

If you don't have too many files, you could probably just spawn a separate pipeline for each. eg:

for i in *; do tail -f "$i" | sed "s@^@$i: @" & done; wait

That omits the filter, so maybe:

for i in *; do tail -f "$i" | sed -e '/str1/d' -e '/str2/d' -e "s@^@$i: @" & done; wait

This runs a tail -f pipeline in the background for each file, piping the output to a sed instance which prepends each line with the file name. You'll want to clean up the processes afterwords. If this is in a script, record the pids and forward any signals to them, otherwise, just clean up with pkill when you're done.

You can also just do:

tail -f *.log | awk '/^==>[^<]*<==$/ {fname = $2; next } ! /str1/ && ! /str2/ {printf "%s: %s\n", fname, $0}'

which seems a bit cleaner and has fewer issues with potentially interleaved output, but I would worry a bit more about buffering issues and the match on the filename is a bit fragile.

Upvotes: 3

Related Questions