CODEWITHSUNDEEP
bashfor-loopubuntu-14.04background-process
combinatoricky
combinatoricky

Reputation: 175

Background processes in bash for loop

On Ubuntu 14.04, I have some Python scripts that I need to run on a range of inputs. Some of these can be run in parallel, so I launch them as background processes from a bash script (below).

for m in `seq 3 7`;
do
        python script_1.py $m
        for a in `seq 1 10 201`;
        do
                for b in `seq 0 9`;
                do
                        t=$(($a + $b))
                        python script_2.py $m $t &
                done
                wait
        done
done

So I would like to run the Python script in batches of 10, then wait until the entire batch has finished before moving on to the next batch of 10, hence the wait command.

However, I find that when I run this bash script, script_2.py runs on the first 20 input values, rather than just the first 10, as background processes. Moreover, the script continues to execute as desired, in batches of 10, after this first batch of 20. Is there an obvious reason why this is happening, and how I can prevent it from doing so?

Cheers!

Upvotes: 2

Views: 4272

Answers (1)

mauro
mauro

Reputation: 5950

I don't see anything wrong in your code. The only possible explanation that comes to my mind is that the first 10 executions of your script_2.py exit almost immediately so you have the impression that 20 instances are executed in parallel the very first time. I'd add some debug code to your script to check this. Something like:

...
for b in {0..9} ; do
    t=$(($a + $b))
    echo "now running script_2.py with t=${t}" >> mylog.txt
    python script_2.py $m $t &
done
echo "now waiting..." >> mylog.txt
wait
...

Upvotes: 2

Related Questions