CODEWITHSUNDEEP
multithreadingperl
Josquin
Josquin

Reputation: 1

Error using perl module Parallel::ForkManager

I am getting the following error message when I am using the perl module Parallel::ForkManager, and I can't figure out what's wrong in my code.

What's weird is that the program run fine for the first 70 bam files it is sending on the different threads and then it output me the error bellow.

Any help would be so appreciate.

Thanks a lot. :)

Here is the errors:

Cannot fork: Cannot allocate memory at /usr/local/perl-5.24.0/lib/site_perl/5.24.0/Parallel/ForkManager.pm line 52, line 8260.

The storable module was unable to store the child's data structure to the temp file "/tmp/1341650.1.longjob.q/dZnCgF1MiR/Parallel-ForkManager-28852-28855.txt": can't create /tmp/1341650.1.longjob.q/dZnCgF1MiR/Parallel-ForkManager-28852-28855.txt: No such file or directory at /usr/local/perl-5.24.0/lib/site_perl/5.24.0/Parallel/ForkManager.pm line 84.

Here is my code:

if ($threads == 1) {
    for my $id (0 .. $nbInputFiles) {
        $self->{$id}->{_currentFile} = $self->{inputFiles}[$id] ;
        $self->readBamInputFile($id) ;
        $self->correctMatrice($id) ;
        print STDERR "INFO $prog $datestring: $self->{$id}->{_currentFile} -> correcting matrice done !\n" ;
    }
}
else {
    my $pm = new Parallel::ForkManager($threads);
    $pm->set_waitpid_blocking_sleep(0);

    $pm->run_on_finish(sub {
        my ($pid, $exit, $id, $signal, $core, $data) = @_;

        $self->{$id}->{cov} = $data->{ret}->{$id}->{cov} ;
        $self->{$id}->{_currentSample} = $data->{ret}->{$id}->{_currentSample} ;
        $self->correctMatrice($id) ;
        print STDERR "INFO $prog $datestring: $data->{ret}->{$id}->{_currentFile} -> correcting matrice done !\n" ;

        $ret[$id] = delete $data->{ret};
        $err[$id] = delete $data->{err};
    });

    for my $id (0 .. $nbInputFiles) {
        $self->{$id}->{_currentFile} = $self->{inputFiles}[$id] ;
        $pm->start($id) and next;

        # here do the child
        # my $res = eval { $task[0]->($self) };
        my $res = eval { &readBamInputFile($self, $id) };
        $pm->finish(0, { ret => $self, err => $@ });
    }
    $pm->wait_all_children;
}

Upvotes: 0

Views: 1502

Answers (2)

Sathya Babji
Sathya Babji

Reputation: 11

Wrapping it in array ref worked for me.

$pm->finish(0, [{x=>1}]);

Upvotes: 1

ikegami
ikegami

Reputation: 385789

It sounds like the /tmp drive is getting full, or there's some permission issue, or some process is cleaning it up behind your back.

Try using another directory for the temporary file.

To do that, set the environment variable TMPDIR, or pass the name of the directory as a second argument to the constructor:

Parallel::ForkManager->new($max_process, $tmp_dir_qfn)

Upvotes: 0

Related Questions