CODEWITHSUNDEEP
greppipeio-buffering
Enlico
Enlico

Reputation: 28384

What determines whether grep buffers the output or not?

If I execute

$ echo XXX | grep XXX

I immediately see the XXX output, because, I understand, there is an EOF passed to grep.

However, if I do something like this,

cat <(yes 'a' | head -c 4090) /dev/stdin > /dev/stdout | sed ';' | grep XXX

and then hit XXXEnter, I don't get any output yet. I have to enter 3 more bytes, e.g. via hitting Enter 3 more times, to get the output.

This reveals that grep is holding on to 4096 character before flushing the output.

However, it appears that having that dummy sed ';' process in the middle is necessary to observe this behavior. In other words, if the pipe was just

cat <(yes 'a' | head -c 4090) /dev/stdin > /dev/stdout | grep XXX

then the output would be visible upon pressing XXXEnter.

Where does this difference stem from?

Upvotes: 2

Views: 58

Answers (1)

KamilCuk
KamilCuk

Reputation: 140880

This reveals that grep is holding on to 4096 character before flushing the output.

No, that reveals that sed is holding the characters. You can use sed -u to make it unbuffered.

What determines whether grep buffers the output or not?

Typically in prevalent most cases whether the standard output file descriptor refers to a terminal device or not. isatty(1) returns true -> output is line buffered, isatty(1) return false -> output is fully buffered.

See https://www.gnu.org/software/libc/manual/html_node/Buffering-Concepts.html :

Newly opened streams are normally fully buffered, with one exception: a stream connected to an interactive device such as a terminal is initially line buffered.

Also I find those /dev/stdin >/dev/stdout confusing. Check

cat <(yes 'a' | head -c 4090) <(sleep 1; echo XXX; sleep 1; echo XXX) | sed ';' | grep XXX

vs

cat <(yes 'a' | head -c 4090) <(sleep 1; echo XXX; sleep 1; echo XXX) | sed -u ';' | grep XXX

Upvotes: 4

Related Questions