CODEWITHSUNDEEP
linuxbashsshparallel-processingbackground-process
Atlas
Atlas

Reputation: 153

Wait for parallel remote process to finish bash ssh

I am trying to run a script on several linux machines in the background. My bash script looks like this:

for i in {1..1000}; do  
  for j in {1..20}; do      
    ssh -n -f remotehost$j "sh -c 'cd /blah/; nohup ./script.sh $i > /dev/null 2>&1 &'"
    NPROC=$(($NPROC+1))
    if [ "$NPROC" -ge 40 ]; then
        echo "Waiting for work to finish"
        wait
        NPROC=0
    fi        
  done    
done

This is my attempt to limit the load on the servers and only have a net of 40 processes run over 20 hosts at any given time. This does not work though and all processes start at the same time. Could you please guide me on how to either wait for remote background process to finish or to prevent more than n processes from starting up on a given remote host.

Upvotes: 2

Views: 3119

Answers (6)

damienfrancois
damienfrancois

Reputation: 59320

A parallel SSH client (pdsh, cssh, pssh, etc) would be the way to go.

Upvotes: 0

Ole Tange
Ole Tange

Reputation: 33748

With GNU Parallel you can do this to run 40 jobs on every of the 20 machines:

parallel -j20 seq 1000 \| parallel -j40 -I I ssh server{} \''"cd /blah/; ./script.sh I"'\' ::: {1..20}

The 10 seconds installation will try do to a full installation; if that fails, a personal installation; if that fails, a minimal installation:

wget -O - pi.dk/3 | bash

or:

curl pi.dk/3/ | bash

Watch the intro video for a quick introduction: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). You command line with love you for it.

Upvotes: 2

konsolebox
konsolebox

Reputation: 75588

Checkout GNU Parallel as well: http://www.gnu.org/software/parallel/.

Upvotes: 1

Aleks-Daniel Jakimenko-A.
Aleks-Daniel Jakimenko-A.

Reputation: 10673

You can use xargs to limit the number of processes:

maxPerServer=2
for j in {1..20}; do
    echo -n {1..1000} | xargs -P "$maxPerServer" -d ' ' ssh -n -f remotehost$j "sh -c 'cd /blah/; ./script.sh {} > /dev/null 2>&1 '" &
done

This code will run max 2 processes per server.

Upvotes: 0

mvp
mvp

Reputation: 116427

You should use tools that were designed specifically for this purpose. I use pssh (aka parallel ssh), and it works pretty well.

Also, it is widely available in common Linux repositories, for example in Ubuntu you can install it using sudo apt-get install pssh.

Upvotes: 1

Barmar
Barmar

Reputation: 782508

Run the remote processes in the foreground, and put ssh in the background:

ssh -n -f remotehost$j "sh -c 'cd /blah/; ./script.sh $i > /dev/null 2>&1 '" &

Upvotes: 0

Related Questions