CODEWITHSUNDEEP
phprabbitmqmessage-queuemqbeanstalkd
Craig Francis
Craig Francis

Reputation: 1935

PHP Message Queue for low volume jobs

When using PHP and a Message Queue (MQ) system like RabbitMQ, Beanstalkd, etc.

Adding jobs to the queue seems simple enough (often a HTTP request).

But to process the queued jobs, most examples involve a PHP script with a while loop, where a connection is established to the MQ server, and this eventually responds with a job (either immediately, if one is already waiting, or the connection will wait until a job is added).

Unfortunately PHP is not ideal for long running jobs, it will often hit different timeouts, and memory management isn't exactly a strong point (it's fairly easy to create code that does not clean up its memory after completing each job).

So I'm wondering if there is a better way to wait for the next job to be made available (potentially waiting many hours), and then it starts the execution of the PHP script as needed (maybe the stdout from the script notes if it was successful or not).

It might be as simple as a bash script?

It could use curl to collect the job (example), and when it has a response, it starts/runs the php script.

I suspect it will need to have a lock, so multiple instances of it cannot be run at the same time, maybe using systemd or a cron job (every 15 min?) to restart it if anything was to fail (or server restart).

I could write this, but would rather use something that's known to work well already.

Upvotes: 0

Views: 763

Answers (1)

Maxim Fateev
Maxim Fateev

Reputation: 6890

Check out Roadrunner PHP application server. It supports long runninug PHP scripts for exactly the reasons you outlined.

It has out of the box integration with many existing queueing systems including RabbitMQ and Kafka.

Upvotes: 0

Related Questions