Bash: Copy Contents of Log File to another file just-in-time / simultaneously

Question

I have to use an application, which is logging to a fixed file ("foo.log"). However, I have to produce my own log file, enriched additional information ("bar.log"). Hence, I need to copy the contents of foo.log to bar.log.

The easiest approach would be to wait for the application to finish and then simply copy / cat contents from foo.log to bar.log. However, is there a way to copy contents from foo.log to bar.log just-in-time, i.e. whenever a new line was written to foo.log?

Thanks in advance.

Answer 1

You could

tail -f -c +0 foo.log

or

while read -r line ;do
     # do something with "$line"
   done < <(tail -f -c +0 foo.log)

or better

exec {logfile}< <(tail -f -c +0 foo.log)
while :; do
    while read -t 0 -u $logfile _ &&
          read -u $logfile line ;do
        echo do something with $line
    done
    read -sn1 -t .2 -r key && break
done

If you hit any key, loop will exit, but you could re-run loop (without exec step), then follow rest of log from last interrupt.

followLogFile() {
    while :; do
        while read -t 0 -u $logfile _ &&
              read -u $logfile line ;do
            echo do something with $line
        done
        read -sn1 -t .2 -r key && break
    done
}

Sample

First open another window, then hit:

cd $(mktemp -d)
pwd
msgs=(foo bar baz)
while :;do
    echo $((i++)) ${msgs[RANDOM%3]}
    sleep $((RANDOM%2)).$RANDOM
done >foo.log

Then is 1st window (with answer of pwd into $tempDir):

followLogFile() {
    while :; do
        while read -t 0 -u $logfile _ &&
              read -u $logfile line ;do
            case $line in
                *foo* ) echo "Foo found in log: '$line'" ;;
                * ) echo "Doing something else with '$line'" ;;
            esac
        done
        read -sn1 -t .2 -r key && break
    done
}

exec {logfile}< <(tail -f -c +0 $tempDir/foo.log)

followLogFile

could produce somehing like

Doing something else with '0 baz'
Doing something else with '1 bar'
Doing something else with '2 bar'
Foo found in log: '3 foo'
Doing something else with '4 bar'
Doing something else with '5 baz'
Doing something else with '6 bar'
Doing something else with '7 bar'
Doing something else with '8 bar'
Foo found in log: '9 foo'
Doing something else with '10 bar'
Doing something else with '11 bar'
Doing something else with '12 bar'
Foo found in log: '13 foo'

Then if you hit any key, this will stop, but if you

followLogFile

This will continue:

Foo found in log: '14 foo'
Foo found in log: '15 foo'
Foo found in log: '16 foo'
Doing something else with '17 bar'
Doing something else with '18 baz'
Foo found in log: '19 foo'

...

Closing $logfile

Once done, you could:

exec {logfile}<&-

But this will be done when your script exit.