CODEWITHSUNDEEP
multithreadingperl
thc
thc

Reputation: 9705

Keeping track of background processes?

I want to run a bunch of processes called by Perl, but only want to run 8 at a time (the number of cores on my machine). For example:

for ( my $i = 1; $i <= 20; $i++) {
        my $command = `some_process $i &`;
}

That would run 20 threads of the process, but I would run out of CPU and/or memory. I want to run 8 to start with, and then when a process finishes, launch another one until 20 are completed.

What is an easy way to do this?

Upvotes: 2

Views: 189

Answers (2)

Kexy Knave
Kexy Knave

Reputation: 1

I would suggest looking at the threads perldoc http://perldoc.perl.org/threads.html

Count your threads. When one finishes, then make a new thread and just keep the # of threads <= 8.

This may not equally distribute the load, but it will let you run several concurrent tasks, and you can limit how many you want to run.

Upvotes: 0

flaviodesousa
flaviodesousa

Reputation: 7855

You could try using wait bultin (see perlfunc(1) manpage):

for ( my $i = 1; $i <= 20; $i++) {
  wait if $i > 8;
  my $command = `some_process $i &`;
}

But there is no guarantee the processes will be distributed evenly among the processors.

Upvotes: 2

Related Questions