CODEWITHSUNDEEP
ssh
user3502260
user3502260

Reputation: 220

SSH to multiple hosts at once

I have a script which loops through a list of hosts, connecting to each of them with SSH using an RSA key, and then saving the output to a file on my local machine - this all works correctly. However, the commands to run on each server take a while (~30 minutes) and there are 10 servers. I would like to run the commands in parallel to save time, but can't seem to get it working. Here is the code as it is now (working):

for host in $HOSTS; do
    echo "Connecting to $host"..
    ssh -n -t -t $USER@$host "/data/reports/formatted_report.sh"
done

How can I speed this up?

Upvotes: 5

Views: 14064

Answers (4)

Doug Judd
Doug Judd

Reputation: 400

The Hypertable project has recently added a multi-host ssh tool. This tool is built with libssh and establishes connections and issues commands asynchronously and in parallel for maximum parallelism. See Multi-Host SSH Tool for complete documentation. To run a command on a set of hosts, you would run it as follows:

$ ht ssh host00,host01,host02 /data/reports/formatted_report.sh

You can also specify a host name or IP pattern, for example:

$ ht ssh 192.168.17.[1-99] /data/reports/formatted_report.sh
$ ht ssh host[00-99] /data/reports/formatted_report.sh

It also supports a --random-start-delay <millis> option that will delay the start of the command on each host by a random time interval between 0 and <millis> milliseconds. This option can be used to avoid thundering herd problems when the command being run accesses a central resource.

Upvotes: 0

Anand
Anand

Reputation: 355

Try massh http://m.a.tt/er/massh/. This is a nice tool to run ssh across multiple hosts.

Upvotes: 0

user3502260
user3502260

Reputation: 220

I tried using & to send the SSH commands to the background, but I abandoned this because after the SSH commands are completed, the script performs some more commands on the output files, which need to have been created.

Using & made the script skip directly to those commands, which failed because the output files were not there yet. But then I learned about the wait command which waits for background commands to complete before continuing. Now this is my code which works:

for host in $HOSTS; do
    echo "Connecting to $host"..
    ssh -n -t -t $USER@$host "/data/reports/formatted_report.sh" &
done
wait

Upvotes: 4

Tomas Pastircak
Tomas Pastircak

Reputation: 2857

You should add & to the end of the ssh call, it will run on the background.

for host in $HOSTS; do
    echo "Connecting to $host"..
    ssh -n -t -t $USER@$host "/data/reports/formatted_report.sh" &
done

Upvotes: 6

Related Questions