CODEWITHSUNDEEP
linuxbash
Fabian Schäfer
Fabian Schäfer

Reputation: 31

Dynamic number of commands parallel execution

I have an array with hostnames and I want to execute an other script with all the hostnames parallel. At first I did it iterative with a for-loop. But I want it parallel and I used the wait-command, but then I can't have dynamic number of hosts.

./sub.sh ${arr[0]} &
./sub.sh ${arr[1]} &
./sub.sh ${arr[2]}
wait

Now I need a function which with I can execute all the elements of the array. Do you have any solution for me?

Upvotes: 1

Views: 335

Answers (2)

Mark Setchell
Mark Setchell

Reputation: 207455

With GNU Parallel like this:

# Generate test data
arr=($(seq 10))

# Check it looks good
echo "${arr[@]}"
1 2 3 4 5 6 7 8 9 10

# Process all elements of the array - just using "echo"
parallel -k echo {} ::: "${arr[@]}"
1
2
3
4
5
6
7
8
9
10

Other useful options, for parallel parameters:

  • -j N will run N jobs at a time
  • --progress will show a progress bar
  • -k will keep the outputs in order
  • --tag will tag outputs (according to host in your case)

In case the above is not clear in how it applies to your specific example, I am suggesting you do:

parallel --tag -k ./sub.sh {} ::: "${arr[@]}"

Upvotes: 3

akiva
akiva

Reputation: 2737

Try the below:

for host in "${arr[@]}" ; do
   ./sub.sh "$host" &
done

# wait for the background processes to finish
wait

Upvotes: 2

Related Questions