Pipe | Redirection < > Precedence

Question

I want to make clear: When does pipe | or redirection < > take precedence in a command?

This is my thought but I need confirmation this is how it works.

Example 1:

sort < names | head

The pipe runs first: names|head then it sorts what is returned from names|head

Example 2:

ls | sort > out.txt

This one seems straight forward by testing, ls|sort then redirects to out.txt

Example 3:

Fill in the blank? Can you have both a < and a > with a | ???

Answer 1

In terms of syntactic grouping, > and < have higher precedence; that is, these two commands are equivalent:

sort < names | head
( sort < names ) | head

as are these two:

ls | sort > out.txt
ls | ( sort > out.txt )

But >, <, and | are not "pure" computations — they have side-effects — so you can't think of them quite like mathematical operations, and in more complicated cases, you have to keep more things in mind than just syntactic grouping; the order in which things happens also matters. For example, these two are not equivalent:

some_command >out.txt 2>&1
some_command 2>&1 >out.txt

Within a simple command, the redirections are performed from left to right; so:

• >out.txt 2>&1 means "first create or truncate out.txt and connect standard output (file descriptor 1) to write to it, then connect standard error (file descriptor 2) to write to the same stream as standard output"; so the standard output and standard error of some_command end up combined in out.txt.
• 2>&1 >out.txt means "first connect standard error to write to the same stream as standard output, then create or truncate out.txt and connect standard output to write to it"; so the standard output of some_command goes to out.txt, but the standard error of some_command ends up going to the standard output of the complete command.

In a pipeline, the pipe connections are performed before the redirections in the individual commands that make up the pipeline, which means that those redirections can supersede the pipe connections.

This means that something like this:

A.sh >A.txt 2>&1 | B.sh   # weird

doesn't make a lot of sense, because A.sh >A.txt 2>&1 doesn't print anything to its standard output, so nothing flows over the pipe to B.sh; whereas something like this:

A.sh 2>&1 >A.txt | B.sh   # OK

makes perfect sense: it sends A.sh's standard output to a file, and pipes A.sh's standard error over to B.sh's standard input.

If you want to try out different permutations for yourself, you might find it helpful to declare some helper functions that help you experiment:

# Print text on both stdout and stderr, with labels
# to help distinguish:
function multiprint() {
    echo "[stdout] $*"
    echo "[stderr] $*" 1>&2
}

# Copy stdin to stdout, with a custom label:
function preprint() {
    local line
    while IFS= read -r line ; do
        echo "[$*] $line"
    done
}

Here are some things you might try out:

multiprint X | preprint Y

multiprint X >/dev/null | preprint Y

multiprint X 3>&1 1>&2 2>&3 | preprint Y

(Or, better yet, make a guess about what they'll print, and then try them out to see if your guess was right!)

Answer 2

This is pretty much what I understand after doing some reading (including ruakh's answer)

First of all, if you redirect multiple times, all the redirections are performed, but only the last redirection will take effect (assuming none of the earlier redirections cause error)

• e.g. cat < in1.txt < in2.txt is equivalent to cat < in2.txt, unless in1.txt does not exist in which case this command will fail (since < in1.txt is performed first)

• Similarly, with cat in.txt > out1.txt > out2.txt, only out2.txt would contain the contents of out2.txt, but since > out1.txt was performed first, out1.txt would be created if it doesn't exist.

What pipe does is connect the stdout of previous command to the stdin of the next command, and that connection comes before any other redirections (from Bash manual).

So you can think of

cat in1.txt > out1.txt | cat > out2.txt

as

cat in1.txt > pipe > out1.txt; cat < pipe > out2.txt

And applying the multiple redirection rule mentioned before, we can simplify this to

cat in1.txt > out1.txt; cat < pipe > out2.txt

Result: The content of in1.txt is copied to out1.txt, since nothing was written to pipe

---

Using another of [ruakh][3]'s example,

cat < in1.txt | cat < in2.txt

is roughly equivalent to

cat > pipe < in1.txt; cat < pipe < in2.txt

which is effectively

cat > pipe < in1.txt; cat < in2.txt

Result: This time something is written to the pipe, but since the second cat reads from in2.txt instead of pipe, only the content of in2.txt is printed out. If the pipe is in the middle of the same side (> or <) redirection, it will be ingored.

Answer 3

It's a little unorthodox, but perfectly legal, to place the < anywhere you like, so I prefer this as it better illustrates the left-to-right data flow:

<input.txt sort | head >output.txt

The only time you cannot do this is with built-in control structure commands (for, if, while).

# Unfortunately, NOT LEGAL
<input.txt  while read line; do ...; done

Note that all of these are equivalent commands, but to avoid confusion you should use only the first or the last one:

<input.txt grep -l foobar
grep <input.txt -l foobar
grep -l <input.txt foobar
grep -l foobar <input.txt

Because the file name must always come directly after the redirection operator, I prefer to leave out the optional space between the < and the file name.

Answer 4

From man bash (as are the other quotes):

SHELL GRAMMAR
   Simple Commands
       A simple command is a sequence of optional variable assignments followed by
       blank-separated words and redirections, and terminated  by  a  control
       operator. The first word specifies the command to be executed, and is
       passed as argument zero.  The remaining words are passed as arguments
       to the invoked command.

       The return value of a simple command is its exit status, or 128+n if
       the command is terminated by signal n.

   Pipelines
       A pipeline is a sequence of one or more commands separated by one of
       the control operators | or |&.  The format for a pipeline is:

              [time [-p]] [ ! ] command [ [|⎪|&] command2 ... ]

In other words, you can have any number of redirections for a (simple) command; you can also use that as part of a pipeline. Or, put another way, redirection binds more tightly than pipe.

There are a couple of ways to get work around this (although they're rarely either necessary or aesthetic):

1. You can make a "compound command" and redirect into it:

 Compound Commands
   A compound command is one of the following:

   (list)  list is executed in a subshell environment (see
           COMMAND EXECUTION ENVIRONMENT below).  Variable
           assignments  and  builtin  commands  that  affect  the
           shell's environment do not remain in effect after the
           command completes.  The return status is the exit status of list.

   { list; }
          list  is  simply  executed  in the current shell environment.  list
          must be terminated with a newline or semicolon.  This is known as a
          group command. The return status is the exit status of list.  Note
          that unlike the metacharacters ( and ), { and } are reserved words
          and must occur where a reserved word is permitted to be recognized.
          Since they do not cause a word break, they must be separated from
          list by whitespace or another shell metacharacter.

So:

$ echo foo > input
$ { cat | sed 's/^/I saw a line: /'; } < input
I saw a line: foo

2. You can redirect to a pipe using "process substitution":

Process Substitution
   Process  substitution  is  supported on systems that support named pipes
   (FIFOs) or the /dev/fd method of naming open files.  It takes the form of
   <(list) or >(list).  The process list is run with its input or output
   connected to a FIFO or some file in /dev/fd.  The name of this file is
   passed as  an  argument  to  the  current  command  as the result of the
   expansion.  If the >(list) form is used, writing to the file will provide
   input for list.  If the <(list) form is used, the file passed as an argument
   should be read to obtain the output of list.

So:

 rici@...$ cat > >(sed 's/^/I saw a line: /') < <(echo foo; echo bar)
 I saw a line: foo
 rici@...$ I saw a line: bar

(Why the prompt appears before the output terminates, and what to do about it are left as exercises).

Answer 5

Corrections:

Example 1:

sort < names | head

In this case, input redirect runs first (names are sorted), then the result of that is piped to head.

In general you can read from left to right. The standard idiom works as follows:

• Use of input redirection "<" tells the program reads from a file instead of stdin
• Use of output redirection ">" tells the program to output to a file instead of stdout
• Use of pipe "program_a | program_b" takes everything that would normally be output by program_a to stdout, and feeds it all directly to program_b as if it was read from stdin.