CODEWITHSUNDEEP
phplinux
xetrill
xetrill

Reputation: 9

linux worker script/queue (php)

I need a binary/script (php) that does the following.

Start n process of X in the background and maintain the number processes.

An example:

P.S.: I posted the same question on SV, which makes me probably very unpopular.

Upvotes: 0

Views: 1666

Answers (6)

Maksim Kotlyar
Maksim Kotlyar

Reputation: 3937

I have to solutions to propose. Both do child process reboot on exit, do child process reloading on USR1 signal, wait for the children exit on SIGTERM and so on.

The first is based on swoole php extension. It is very performant, async, non-blocking. Here's the usage example code:

<?php
use Symfony\Component\Process\PhpExecutableFinder;

require_once __DIR__.'/../vendor/autoload.php';
$phpBin = (new PhpExecutableFinder)->find();
if (false === $phpBin) {
    throw new \LogicException('Php executable could not be found');
}
$daemon = new \App\Infra\Swoole\Daemon();

$daemon->addWorker(1, $phpBin, [__DIR__ . '/console', 'quartz:scheduler', '-vvv']);

$daemon->addWorker(3, $phpBin, [__DIR__ . '/console', 'enqueue:consume', '--setup-broker', '-vvv']);

$daemon->run();

The daemon code is here

Another is based on Symfony process library. It does not require any extra extensions. The usage example and daemon code could be found here

Upvotes: 0

Jeff Ferland
Jeff Ferland

Reputation: 18282

Here's something in Perl I have in my library (and hey, let's be honest, I'm not going to rig this up in PHP just to give you something working in that language this moment. I'm just using what I can copy / paste).

#!/usr/bin/perl
use threads;
use Thread::Queue;

my @workers;
my $num_threads = shift;
my $dbname = shift;
my $queue = new Thread::Queue;

for (0..$num_threads-1) {
        $workers[$_] = new threads(\&worker);
                print "TEST!\n";
}

while ($_ = shift @ARGV) {
        $queue->enqueue($_);
}

sub worker() {
        while ($file = $queue->dequeue) {
                system ('./4parser.pl', $dbname, $file);
        }
}

for (0..$num_threads-1) { $queue->enqueue(undef); }
for (0..$num_threads-1) { $workers[$_]->join; }

Whenever one of those systems calls finishes up, it moves on dequeing. Oh, and damn if I know hwy I did 0..$numthreads instead of the normal my $i = 0; $i < ... idiom, but I did it that way that time.

Upvotes: 1

WebDevEric
WebDevEric

Reputation: 11

Have you tried making a PHP Daemon before?

http://kevin.vanzonneveld.net/techblog/article/create_daemons_in_php/

Upvotes: 1

Dooltaz
Dooltaz

Reputation: 2463

I would go in the direction that andres suggested. Just put something like this at the top of your pro.php file... 

$this_file = __FILE__;
$final_count = 50;

$processes = `ps auwx | grep "php -f $this_file"`;
$processes = explode("\n", $processes);
if (count($processes)>$final_count+3) {
        exit;
}
//... Remaining code goes here

Upvotes: 1

Dennis Williamson
Dennis Williamson

Reputation: 359845

Pseudocode:

for (i=1; i<=50; i++)
  myprocess
endfor

while true
  while ( $(ps --no-headers -C myprocess|wc -l) < 50 )
    myprocess
  endwhile
endwhile

If you translate this to php and fix its flaws, it might just do what you want.

Upvotes: 1

andres descalzo
andres descalzo

Reputation: 14967

Can you use the crontab linux and write to a db or file the number of current process?. If DB, the advantage is that you can use to procedure and lock the table, and write the number of process.

But to backgroun you should use & at the end of the call to script 

# php-f pro.php &

Upvotes: 1

Related Questions