CODEWITHSUNDEEP
bashsedpipe
HenryM
HenryM

Reputation: 111

How does the piping of sed commands control the flow information in input and output space?

I'm piping several sed commands, but I don't understand how information flows.

Example:

command1 | command 2 | command3

Does piping wait until all of command1 is completed and transferred to the output space before then piping all of output space of command 1 to the input space of command2; or once a line from command1 is transferred to the output space it then immediately piped to the input space of command 2, without waiting for the rest of the output space in command1 to be fully filled out by the remaining lines?

I hope someone can briefly clarify this.

Upvotes: 1

Views: 254

Answers (2)

John Kugelman
John Kugelman

Reputation: 361977

Commands in a pipeline execute in parallel. They're all started at the same time and run concurrently. The right process receives the left process's output as it's generated. It doesn't have to wait until the left process finishes.

There's typically a small buffer associated with the pipe, say 4KB. Processing usually happens in 4KB chunks rather than line by line. Processes can flush their output more frequently if they're explicit about it or if they disable buffering, but that's the default behavior.

If you don't want the programs to run in parallel then you'll want to execute them separately and store their output in temp files.

command1 > cmd1.out
command2 < cmd1.out > cmd2.out
command3 < cmd2.out
rm cmd1.out cmd2.out

Upvotes: 1

Mark Harrison
Mark Harrison

Reputation: 304644

The shell starts three processes and runs them all concurrently. Process 1 (first sed) will send output to process 2 (second sed), which will wait for input to be ready.

In principle this could be replaced by three sequential executions of sed, with temporary files storing the output of one phase for reading by the subsequent phase.

Upvotes: 0

Related Questions