CODEWITHSUNDEEP
bashloggingio-redirection
xApple
xApple

Reputation: 6466

Bash: log stdout and stderr while preserving order and provenance

It's fairly simple to log both the stdout and the stderr of a command to a log file:

./foo.sh &> log.txt

The problem is that when inspecting the log file, one doesn't know anymore which line was coming from which stream. This could be fixed by redirecting stdout and stderr to two separate files, but then the chronology and interleaving of the output is lost.

An other solution would be to redirect to three files. One with the stdout, one with the stderr, and one with both combined. Something like:

./foo.sh 2> >(tee stderr | tee -a combined) 1> >(tee stdout | tee -a combined)

But that is not be very elegant to have so many files (and this command still dumps a copy of the output on the shell).

I found an interesting bash function that would color only stderr messages in red:

color()(set -o pipefail;"$@" 2>&1>&3|sed $'s,.*,\e[31m&\e[m,'>&2)3>&1

but it doesn't preserve the order of the output and the result is unreadable in a text editor. Given the following program for foo.sh:

for i in 1 2; do
   for j in 1 2; do
     printf '%s\n' "out $i"
   done
   for k in 1 2; do
     printf '%s\n' "err $i" >&2
   done
done

Running color ./foo.sh produces:

out 1
out 1
out 2
out 2
[31merr 1[m
[31merr 1[m
[31merr 2[m
[31merr 2[m

How could one easily end up with something such as this in a single log file ?

@| out 1
@| out 1
$| err 1
$| err 1
@| out 2
@| out 2
$| err 2
$| err 2

Upvotes: 3

Views: 1825

Answers (2)

Eric
Eric

Reputation: 79

How about redirecting stderr to ./tmperr and stdout to ./tmpout, now run two scripts in the background that each continuously read a single line from tmperr or tmpout and then remove that line? This is obviously less than ideal, but if there is a decent delay in the actual foo.sh script, this should work out.

Upvotes: 2

Zoltán Haindrich
Zoltán Haindrich

Reputation: 1808

maybe you are looking for script it records both stdout, stderr and the commands...it starts a new shell in which it records everything (or use -c _cmd_)

 $ script tx1

your color() function breaks order because sed is buffering...

Upvotes: 2

Related Questions