CODEWITHSUNDEEP
bashsshstdoutgnu-parallel
Ben
Ben

Reputation: 6887

gnu parallel supresses output of ssh and scp when using --line-buffer option

I am trying to use gnu parallel to run some ssh and scp commands.

When I do NOT use -u or --line-buffer options, I can see the output for each thread after the thread has finished.

Because my ssh commands take a while I want to see output as it occurs.

When I try to use -u tagging does not work.

When I try to use --line-buffer, I don't get any of the output, even after threads have finished.

I've written this simple test command which behaves in the same way:

parallel -q ssh user@{} 'ls && sleep 4 && ls' ::: host1 host2

I have been looking at this very similar question https://unix.stackexchange.com/questions/101360/scp-does-not-display-output-when-used-with-gnu-parallel but the suggested solution of using the script command did not work for me. Might be because my system is not linux. (Am using Solaris).

Any help would be appreciated.

Upvotes: 1

Views: 776

Answers (1)

Ole Tange
Ole Tange

Reputation: 33685

If you want to run the same command on a bunch of machines, use --nonall. https://www.gnu.org/software/parallel/parallel_tutorial.html#Running-the-same-commands-on-all-hosts

We need to determine whether it is due to ssh or it is a general problem. So test these:

parallel --version
parallel --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel --tag --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host1 --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host2 --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host1,host2 --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host1 --tag --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host2 --tag --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c
parallel -S host1,host2 --tag --line-buffer 'echo {}; sleep 4; echo {}' ::: a b c

Have you somehow enabled --compress (this would explain the behaviour)?

Upvotes: 1

Related Questions