Explanation needed for tee, process substitution, redirect...and different behaviors in Bash and Z shell ('zsh')

Question

Recently in my work, I am facing an interesting problem regarding tee and process substitution.

Let's start with examples:

I have three little scripts:

$ head *.sh

File one.sh

#!/bin/bash
echo "one starts"
if [ -p /dev/stdin ]; then
    echo "$(cat /dev/stdin) from one"
else
    echo "no stdin"
fi

File two.sh

#!/bin/bash
echo "two starts"
if [ -p /dev/stdin ]; then
    echo "$(cat /dev/stdin) from two"
else
    echo "no stdin"
fi

File three.sh

#!/bin/bash
echo "three starts"
if [ -p /dev/stdin ]; then
    sed 's/^/stdin for three: /' /dev/stdin
else
    echo "no stdin"
fi

• All three scripts read from standard input and print something to standard output.
• The one.sh and two.sh are quite similar, but the three.sh is a bit different. It just adds some prefix to show what it reads from the standard input.

Now I am going to execute two commands:

1: echo "hello" | tee >(./one.sh) >(./two.sh) | ./three.sh

2: echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null

First in Bash and then in Z shell (zsh).

Bash (GNU bash, version 5.0.17(1))

$ echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh

three starts
stdin for three: hello
stdin for three: one starts
stdin for three: two starts
stdin for three: hello from two
stdin for three: hello from one

• Why are the outputs of one.sh and two.sh mixed with the origin "hello" and passed to three.sh? I expected to see the output of one and two in standard output and only the "hello" is going to pass to three.sh.

Now the other command:

$ echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null

one starts
two starts
three starts
stdin for three: hello
hello from two
hello from one
<---!!!note here I don't have prompt unless I press Enter or Ctrl-c)

• I redirect all standard output to /dev/null. Why do I see all output from all process substitution this time? Does it seem this behavior conflict with the one above?
• Why don't I have the prompt after having executed the command?
• Why does the command start in order one->two->three, but outputs come in 3->2->1? Even if I added sleep 3 in three.sh, the output is always 3-2-1. I know it should have something to do with standard input blocking, but I'd learn the exact reason.

Zsh (zsh 5.8 (x86_64-pc-linux-gnu))

Both commands,

echo "hello" | tee >(./one.sh) >(./two.sh) >(./three.sh) >/dev/null
echo "hello" | tee >(./one.sh) >(./two.sh) |./three.sh

Give the expected result:

one starts
three starts
two starts
hello from two
hello from one
stdin for three: hello

• It works as expected. But the order of the output is random, it seems that Z shell does something non-blocking here, and the order of the output is dependent on how long each script has been running. What exactly leads to the result?

Answer 1

echo "hello"|tee >(./one.sh) >(./two.sh) |./three.sh

There are two possible order of operations for the tee part of the pipeline

First

1. Redirect standard output to a pipe that's connected to ./three.sh's standard input.
2. Set up the pipes and subprocesses for the command substitutions. They inherit the same redirected standard output pipe used by tee.
3. Execute tee.

Second

1. Set up the pipes and subprocesses for the the command substitutions. They share the same default standard output - to the terminal.
2. Redirect tee's standard output to a pipe that's connected to ./three.sh's standard input. This redirection doesn't affect the pipes set up in step 1.
3. Execute tee.

bash uses the first set of operations, zsh uses the second. In both cases, the order you see output from your shell scripts in is controlled by your OS's process scheduler and might as well be random. In the case where you redirect tee's standard output to /dev/null, they both seem to follow the second scenario and set up the subprocesses before the parent tee's redirection. This inconsistency on bash's part does seem unusual and a potential source of subtle bugs.

I can't replicate the missing prompt issue, but that's with bash 4.4.20 - I don't have 5 installed on this computer.