CODEWITHSUNDEEP
stdoutbuffering
bodacydo
bodacydo

Reputation: 79509

How to make output (stdout, stderr) of any shell command unbuffered?

Is there a way to run shell commands without output buffering?

For example, hexdump file | ./my_script will only pass input from hexdump to my_script in buffered chunks, not line by line.

What is a general solution to make the output for any command unbuffered?

Upvotes: 106

Views: 95806

Answers (5)

Binghoo Dang
Binghoo Dang

Reputation: 37

One should use grep or egrep "--line-buffered" options to solve this. no other tools needed.

Upvotes: 2

vroc
vroc

Reputation: 241

You could also use the script command to make the output of hexdump line-buffered (hexdump will be run in a pseudo terminal which tricks hexdump into thinking its writing its stdout to a terminal, and not to a pipe).

# cf. http://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe/
stty -echo -onlcr
script -q /dev/null hexdump file | ./my_script         # FreeBSD, Mac OS X
script -q -c "hexdump file" /dev/null | ./my_script    # Linux
stty echo onlcr

Upvotes: 24

lambshaanxy
lambshaanxy

Reputation: 23072

Try stdbuf, included in GNU coreutils and thus virtually any Linux distro. This sets the buffer length for input, output and error to zero:

stdbuf -i0 -o0 -e0 command

Upvotes: 182

Nordic Mainframe
Nordic Mainframe

Reputation: 28767

AFAIK, you can't do it without ugly hacks. Writing to a pipe (or reading from it) automatically turns on full buffering and there is nothing you can do about it :-(. "Line buffering" (which is what you want) is only used when reading/writing a terminal. The ugly hacks exactly do this: They connect a program to a pseudo-terminal, so that the other tools in the pipe read/write from that terminal in line buffering mode. The whole problem is described here:

The page has also some suggestions (the aforementioned "ugly hacks") what to do, i.e. using unbuffer or pulling some tricks with LD_PRELOAD.

Upvotes: 25

Siu Ching Pong -Asuka Kenji-
Siu Ching Pong -Asuka Kenji-

Reputation: 8213

The command unbuffer from the expect package disables the output buffering:
Ubuntu Manpage: unbuffer - unbuffer output

Example usage:

unbuffer hexdump file | ./my_script

Upvotes: 37

Related Questions