CODEWITHSUNDEEP
multithreadingperl
user1614240
user1614240

Reputation: 107

thread shared perl

i wrote a code and i need to make it multithreaded. Evething works, but every loop repeats 4 times:

use LWP::UserAgent;
use HTTP::Cookies;
use threads;
use threads::shared;

$| = 1;

$threads = 4;
my @groups :shared = loadf('groups.txt');

my @thread_list = ();
$thread_list[$_] = threads->create(\&thread) for 0 .. $threads - 1;
$_->join for @thread_list;
thread();

sub thread
{
    my $url = 'http://www.site.ru/';
    my $response = $web->post($url, Content =>
                    ['st.redirect' => ''
                    ]);
    foreach $i (@groups)
    {

        my $response  = $web->get($i);
        if(!($response->header('Location')))
        {
            ---------;
        }
        else
        {
            ----------;
        }

    }

}

sub loadf {
    open (F, "<".$_[0]) or erroropen($_[0]); 
    chomp(my @data = <F>);
    close F;
    return @data;
}

groups.txt :

http://www.odnoklassniki.ru/group/47357692739634
http://www.odnoklassniki.ru/group/56099517562922

I understand that i need to use threads::shared; but i can't undestand how to use it.

Your post does not have much context to explain the code sections; please explain your scenario more clearly.

Upvotes: 1

Views: 207

Answers (2)

ikegami
ikegami

Reputation: 386551

The problem is that you never remove from @groups, so all threads do all jobs in @groups.

Here's one solution.

use threads;
use Thread::Queue 3.01 qw( );

my $NUM_WORKERS = 4;

sub worker {
   my ($url) = @_;
   ... download the page ...
}

my $q = Thread::Queue->new();
for (1..$NUM_WORKERS) {
   async {
      while (my $url = $q->dequeue()) {
         worker($url);
      }
   };
}

$q->enqueue($_) for loadf('groups.txt');
$q->end();
$_->join() for threads->list;

Upvotes: 6

ysth
ysth

Reputation: 98398

Why do you need to make it threaded? perl does much better using forks in most cases.

That said, your code starts 4 threads, each of which processes everything in @groups. It sounds like that's not what you want to do. If you want @groups to be a queue of work to do, take a look at Thread::Queue (or Parallel::ForkManager).

Upvotes: 3

Related Questions