CODEWITHSUNDEEP
bashparallel-processingwaitsleep
pku
pku

Reputation: 89

Parallel runs in N-process batches in BASH

I have a for loop and I want to process it 4 times in parallel at a time. I tried the following code from the page https://unix.stackexchange.com/questions/103920/parallelize-a-bash-for-loop:

task(){
sleep 0.5; echo "$1";
}
N=4
(
for thing in a b c d e f g; do
   ((i=i%N)); ((i++==0)) && wait
   task "$thing" &
done
)

I have stored the above file as test.sh, the output I get it is as follows:

path$ ./test.sh
a
b
c
d
path$ e
f
g

and the cursor doesn't come back to my terminal after 'g', it waits/ sleeps indefinitely.I want the cursor come back to my terminal and also I don't understand why the output 'e' has my path preceding it, shouldn't the output be displayed as 'a' to 'g' continuously and the code should stop?

Upvotes: 3

Views: 2889

Answers (2)

Mark Setchell
Mark Setchell

Reputation: 207465

It's pretty hard to understand what you want, but I think you want to do 7 things, called a,b,c...g in parallel, no more than 4 instances at a time.

If so, you could try this:

echo {a..g} | xargs -P4 -n1 bash -c 'echo "$1"; sleep 2' {}

That sends the letters a..g into xargs which then starts a new bash shell for each letter, passing one letter (-n1) to the shell as {}. The bash shell picks up the parameter (its first parameter being $1) and echoes it then waits 2 seconds before exiting - so you can see the pause.

The -P4 tells xargs to run 4 instances of bash at a time in parallel.

Here is a little video of it running. The first one uses -P4 and runs in groups of 4, the second sequence uses -P2 and does 2 at a time:

enter image description here

Or, more simply, if you don't mind spending 10 seconds installing GNU Parallel:

parallel -j4 -k  'echo {}; sleep 2' ::: {a..g}

Upvotes: 10

Roman Tkachuk
Roman Tkachuk

Reputation: 3266

If you press enter toy can see than you are in normal shell. If you want to wait batched process before exit the script just add wait at the end of script

Upvotes: 0

Related Questions