CODEWITHSUNDEEP
ubuntusshdistributedwait
Eduardo Bezerra
Eduardo Bezerra

Reputation: 1963

How to wait for multiple remote processes (via ssh) to finish?

My situation is that I have a number of processes that are supposed to start (more or less) at the same time in different remote hosts. A master script, which spawns all remote processes, should wait for all of them to finish, to only then proceed to the next distributed execution. The way I've tried this so far (with no success on Ubuntu Server 12.04, /bin/sh pointing to /bin/bash) was:

#!/bin/bash

for run_input in run1.data run2.data ; do
   for i in `seq 0 10` ; do
      ssh node$i "/path/to/bin $run_input /path/to/node$i.config" &
   done
   wait
done

So, what's wrong with the above approach?

EDIT:

Note that the command is different for each remote host.

Upvotes: 1

Views: 3825

Answers (2)

Eduardo Bezerra
Eduardo Bezerra

Reputation: 1963

Just answering my own question, there is nothing wrong with the way I was doing it (although some might recommend pssh or something similar to be used instead). The way I suggested in the question, a temporary ssh session is created with the remote host, for each command sent. This ssh session remains running on the background and, by simply calling wait, the script should pause until all background child processes are finished - in this case, the ssh sessions.

What I was doing wrong was that I was piping a file into a while loop, like:

cat file.txt | while read line ; do
    ssh node "do_something_with $line" &
done
wait

The problem with the above is that, when piping into a while loop, a sub-shell is created, so the remote ssh sessions created were not children of this script, but of the sub-shell. Therefore, calling wait had no effect whatsoever.

Upvotes: 4

Gilles Quénot
Gilles Quénot

Reputation: 185790

For running parallel ssh at the same time, I recommend using pssh

Doc is there : http://www.theether.org/pssh/docs/0.2.3/pssh-HOWTO.html

Then :

pssh -h /PATH/TO/FILE/WITH/HOSTS command

Upvotes: 1

Related Questions