CODEWITHSUNDEEP
multithreadingperl
Bijan
Bijan

Reputation: 8670

Perl: Run multiple system commands at once

In perl, I have some code like 

my $enter = `curl -s -m 10 http://URL`;

How would I use threading to run this function 10 times at once?

I found this but I am not sure how to use it to set a specific amount of threads

Edit: I guess I misunderstood what Threads::Queue was doing. My original question still stands for simultaneously running multiple commands at once.

Upvotes: 1

Views: 1793

Answers (1)

stevieb
stevieb

Reputation: 9306

You can use fork(). In this example, I use the Parallel::ForkManager module. $max_forks is the number of processes to run simultaneously (set to two for an example), and you'd put your system/curl code after ### add curl logic here, and remove the print() and sleep() example statements from there as well.

#!/usr/bin/perl

use warnings;
use strict;

use Parallel::ForkManager;

my $max_forks = 2;

my $fork = new Parallel::ForkManager($max_forks);

my @urls = (
             'http://perlmonks.org', 
             'http://stackoverflow.com',
             'http://slashdot.org', 
             'http://wired.com',
);

# on start callback

$fork->run_on_start(
    sub {
            my $pid = shift;
            print "Starting PID $pid\n";
    }
);

# on finish callback

$fork->run_on_finish(
    sub {
        my ( $pid, $exit, $ident, $signal, $core) = @_;
        if ($core){
            print "PID $pid core dumped.\n";
        }
        else {
            print "PID $pid exited with exit code $exit " .
                  " and signal $signal\n";
        }
    }
);

# forking code

for my $url (@urls){

    $fork->start and next;

    ### add curl logic here
    print "$url\n";
    sleep(2);

    $fork->finish;
}

$fork->wait_all_children;

Upvotes: 7

Related Questions