CODEWITHSUNDEEP
linuxshellpipestdoutstdin
Return-1
Return-1

Reputation: 2449

Linux piping behaviour

Ok this is merely to satisfy my curiosity and anyone else that might have a similar question. Please bear with the ignorance and lengthy question as its partially a case of "i dont know what i dont know".

Section 1

Assume a fileToFollow.txt that has some arbitrary content.

row1
row2
row3

Executing tail fileToFollow.txt | cat yields the contents of the file as expected.

Executing tail -f fileToFollow.txt | cat will keep on outputing anything that is written into fileToFollow.txt

I imagined piping as getting the output of one program and feeding it into the input of another ( so for example if cat was a C program it would be able to access that input through the main() arguments ).

Question 1: Is that whats going on here, is cat just called everytime tail has an output?

Section 2

I decided to throw grep into the mix setting up the following:

tail -f fileToFollow.txt | grep "whatever" | cat

Clearly cat is not needed here as grep itself does output to the terminal anyway. But given the idea that piping is output from one program to input in another, i was assuming it would. However, in this case no output is displayed in the terminal.

The following of course works fine:

tail -f fileToFollow.txt | grep "whatever"

I have a hunch i am little bit confused as to how piping might actually work and why the cases i presented aren't behaving as i would expect them to.

Any kind of enlightenment is welcome. Thanks a bunch for taking the time.

Upvotes: 0

Views: 72

Answers (2)

Ville Oikarinen
Ville Oikarinen

Reputation: 410

Answer to question 1: No, cat is running as a process all the time but it's blocked reading stdin when there is nothing available. When the process before it in the pipeline (tail) writes new bytes to the pipe, the read call will return and cat is able to process the new data. After that it will read again and block until new data is available.

Upvotes: 2

ensc
ensc

Reputation: 6984

When you pipe into a program, the stdout of the source will usually switch into buffered mode (see man setvbuf()), which means that a certain amount of data (2KiB or 4KiB or so) must be generated before it will be given to write(2).

Giving it out to tty uses line buffered mode so that buffers are flushed after \n.

There exists a stdbuf tool to modify this behavior.

Upvotes: 1

Related Questions