CODEWITHSUNDEEP
bashshellcurl
Donbhupi
Donbhupi

Reputation: 230

How to append constantly changing stderr to a file or stdout

I might not have been able to phrase my search properly but I could not find any way to collect all values of stderr/stdout in cases where it is overwritten every second of the execution.

It's not a problem with commands like top because it writes a fresh page to stdout for per-second-statistics but in certain cases like curl where progress is shown in stderr and it is overwritten every second, I could not find a way to collect all the per-second values of the stderr, which I need for protocol analysis.

To add further detail:

$ curl -o /dev/null --trace-ascii trace --trace-time    http://cachefly.cachefly.net/10mb.test
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
0 10.0M    0 65057    0     0  43039      0  0:04:03  0:00:01  0:04:02 43027^C

$ curl -o /dev/null --trace-ascii trace --trace-time http://cachefly.cachefly.net/10mb.test
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
13 10.0M   13 1376k    0     0   425k      0  0:00:24  0:00:03  0:00:21  425k^C

As you can see, stderr is being displayed on stdout but is overwritten every second. Even if I append it to a file, the file only contains the last value before the command was terminated whereas I want the file to contain a list of all per-second values. I've read every line of man curl to see if any such thing is possible but maybe that's not where I should be looking. Any help/redirection would be appreciated!

Thanks in anticipation.

Upvotes: 1

Views: 432

Answers (1)

choroba
choroba

Reputation: 241748

The output contains the $'\r' character which moves the cursor back to the beginning of line. Just replace it with newlines in the file and you'll see all the lines.

sed -i~ 's/\r/\n/g' input

sed needs to read the whole "line" to produce output, so if you don't want to wait till the end, switch to Perl:

curl -o /dev/null --trace-ascii trace --trace-time http://cachefly.cachefly.net/10mb.test 2>&1 \
    | perl -pe 'BEGIN{$/="\r"}s/\r/\n/'

Upvotes: 2

Related Questions