CODEWITHSUNDEEP
shellloggingpipestdoutinit
Iniesta8
Iniesta8

Reputation: 409

How to prepend stdout and stderr output with timestamp when redirecting into log files?

In Linux I'm starting a program called $cmd in an init script (SysVInit). I'm already redirecting stdout and stderr of $cmd into two different logfiles called $stdout_log and $stderr_log. Now I also want to add a timestamp in front of every line printed into the logfiles.

I tried to write a function called log_pipe as follows:

log_pipe() {
    while read line; do
        echo [$(date +%Y-%m-%d\ %H:%M:%S)] "$line"
    done
}

then pipe the output of my script into this function and after that redirect them to the logfiles as follows:

$cmd | log_pipe >> "$stdout_log" 2>> "$stderr_log" &

What I get is an empty $stdout.log (stdout) what should be okay, because the $cmd normally doesn't print anything. And a $stderr.log file with only timestamps but without error texts.

Where is my faulty reasoning?

PS: Because the problem exists within an init script I only want to use basic shell commands and no extra packages.

Upvotes: 3

Views: 2977

Answers (4)

Mirandole
Mirandole

Reputation: 21

This was the easiest way for me to accomplish this with timestamp included :

{ ./script.sh | ts >>script-stdout.log; } 2>&1 | ts >>script-stderr.log

Upvotes: 0

Iniesta8
Iniesta8

Reputation: 409

This works, but my command has to run in background because it is within an init script, therefore i have to do:

({ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log) &
echo $! > "$pid_file"

right?

But I think in this case the pid in the $pid_file is not the pid of $cmd...

Upvotes: 0

John1024
John1024

Reputation: 113864

In any POSIX shell, try:

{ cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log

Also, if you have GNU awk (sometimes called gawk), then log_pipe can be made simpler and faster:

log_pipe() {  awk '{print strftime("[%Y-%m-%d %H:%M:%S]"),$0}'; }

Example

As an example, let's create the command cmd:

cmd() { echo "This is out"; echo "This is err">&2; }

Now, let's run our command and look at the output files:

$ { cmd | log_pipe >>stdout.log; } 2>&1 | log_pipe >>stderr.log
$ cat stdout.log
[2019-07-04 23:42:20] This is out
$ cat stderr.log
[2019-07-04 23:42:20] This is err

The problem

cmd | log_pipe >> "$stdout_log" 2>> "$stderr_log"

The above redirects stdout from cmd to log_pipe. The stdout of log_pipe is redirected to $stdout_log and the stderr of log_pipe is redirected to $stderr_log. The problem is that the stderr of cmd is never redirected. It goes straight to the terminal.

As an example, consider this cmd:

cmd() { echo "This is out"; echo "This is err">&2; }

Now, let's run the command:

$ cmd | log_pipe >>stdout.log 2>>stderr.log
This is err

We can see that This is err is not sent to the file stderr.log. Instead, it appears on the terminal. It is never seen by log_pipe. stderr.log only captures error messages from log_pipe.

Upvotes: 4

UtLox
UtLox

Reputation: 4154

In Bash, you can also redirect to a subshell using process substitution:

logger.sh

#!/bin/bash
while read -r line; do
   echo "[$(date +%Y-%m-%d\ %H:%M:%S)] $line"
done

redirection

cmd > >(logger.sh > stdout.log) 2> >(logger.sh > stderr.log)

Upvotes: 1

Related Questions