CODEWITHSUNDEEP
linuxbash
fundamental
fundamental

Reputation: 133

Passing stdout twice to the same command

Hello I am interested in how to pass stdout twice to the same command...

For instance, If I run the following:

seq 5 >a
tac a >b
paste a b

then I get:

1       5
2       4
3       3
4       2
5       1

The following also deliver the same result:

paste <(seq 5) <(seq 5 |tac)

or

seq 5 | paste - <(seq 5 |tac)

I want to use seq 5 only once - perhaps similar to these attempts, which do not work:

seq 5 | paste - <(tac -)

or

seq 5 | tee >(tac 1>&3) | paste - >(3>&1 cat)

I expect some kind of file descriptor manipulation or further process substitution magic will do the trick but I am having trouble getting there.

Upvotes: 2

Views: 476

Answers (4)

chepner
chepner

Reputation: 532398

A simple, symmetrical example would be to use two named pipes.

 trap 'rm p1 p2' ERR EXIT
 mkfifo p1 p2

Then you can have tac filter the output of tee into p2

 seq 5 | tee p1 <(tac > p2) > /dev/null &
 paste p1 p2

or filter the contents of p2 before passing to paste.

seq 5 | tee p1 p2 > /dev/null
paste p1 <(tac p2)

Upvotes: 0

Adrian
Adrian

Reputation: 675

The problem you're facing is that each element in a pipeline is executed in a separate subshell, so any newly-redirected FDs by the elements aren't reflected in the parent shell, which is doing all the I/O coordination.

Some other guru might come up with a magic redirect incantation to solve this problem at the parent level, but for clarity, I'd suggest named pipes (a.k.a. FIFOs) instead. Since they exist outside of the shell environment, they're visible to all pipeline elements at all times.

More importantly, with proper naming, "connecting the pipes" becomes a relatively bug-free process, especially as your pipeline "network" gets more complex. No guessing games about which FD carries which output.

For your example, this works seamlessly:

#!/bin/bash
# Make a temp dir...
tmpd=$(mktemp -d)
# ...and clean it up automatically
trap 'rm -fr $tmpd' EXIT

# Create the FIFO...
mkfifo $tmpd/tac.fifo
# ...and PROFIT!
seq 5 | tee >(tac >$tmpd/tac.fifo) | paste - $tmpd/tac.fifo

Further Reading:

Upvotes: 0

Ondrej K.
Ondrej K.

Reputation: 9679

You could do something like this:

rootdir=$(mktemp -d)  # Strictly speaking not needed, but we create us a tempdir
mkfifo "$rootdir/pipe"  # create a named pipe
seq 5 | tee "$rootdir/pipe" | paste - <(tac "$rootdir/pipe")
# We could now tee (split) output into that pipe
rm -Rf "$rootdir"  # cleanup

You could also have extra file descriptor involved, but you'd still need something to underpin it with to connect both ends of the stream. Problem is, while you could multiplex the output, on a newly created problem you would still only have one input stream (when piping commands connected to stdout of the preceding process at that). A second transmission path must be added.

tmpfile=$(mktemp); exec 3<>"$tmpfile"; rm "$tmpfile"

Creates a temporary file and opens fd 3 for reading/writing. Once we have a file descriptor, we no longer need name for that file in the file system.

seq 5 | tee /proc/self/fd/3 | paste - <(tac </proc/self/fd/3)

We still use tee to split the output between stdout and fd 3. File descriptors are inherited by child process, so we can refer to them with /proc/self/ (also from perspective of tee and tac).

Upvotes: 1

tripleee
tripleee

Reputation: 189936

There is no way to duplicate a single stream, short of using a tool such as tee. A common arrangement is to use a temporary file.

#!/bin/bash
t=$(mktemp -t pastepaste.XXXXXXXXXX) || exit
trap 'rm -f "$t"' ERR EXIT
tee "$t" |
paste - <(tac "$t")

Upvotes: 1

Related Questions