CODEWITHSUNDEEP
phplaravelbeanstalkd
Andrew Brown
Andrew Brown

Reputation: 5424

Multiple Sites on same Beanstalkd Queue

I currently have a couple sites setup on a server, and they all use beanstalkd for their queues. Some of these sites are staging sites. While I know it would be ideal to have the staging site on another server, it doesn't make financial sense to spin up another server for it in this situation.

I recently ran into a very confusing issue where I was deploying a staging site, which included a reseeding of the database. I have an observer setup on some model saves that will trigger a job to be queued, which usually ends up sending out an email. The staging site does not actually have any queue workers setup to run them, but the production site (on the same server) does have the queue workers running.

What appeared to be happening is the staging site was generating the queue jobs, and the production site was running those queue jobs! This causes random users of mine to be spammed with email, since it would serialize the model from staging, and when it unserialized it running the job, it actually matched up with an actual production user.

It seems like it would be very common to have multiple sites on a server running queues, so I'm curious if there is a way to avoid this issue. Elasticsearch has the concept of a 'cluster', so you can run multiple search 'clusters' on one server. I'm curious if beanstalkd or redis or any other queue provider have this ability, so we don't have crosstalk between completely separate websites.

Thanks!

Upvotes: 2

Views: 1034

Answers (2)

Andi Li
Andi Li

Reputation: 1

This is an old question but have you tried running multiple beanstalkd daemons? Simply bind to another port.

Example: beanstalkd -p 11301 &

Use & to fork into background.

Upvotes: 0

Alex Howansky
Alex Howansky

Reputation: 53626

Beanstalkd has the concept of tubes:

Tubes are job queues.

A common use case of tubes would be to have completely different sets of producers and consumers running through a single beanstalk instance such that a given consumer will not know what to do with jobs produced by some of the producers. Producer1 can enqueue jobs into Tube1 and Consumer1 can pick up the jobs from there completely independently of what Producer2 and Consumer2 are doing with Tube2, for example.

For example, if you're using pheanstalk, the producer will call useTube():

$pheanstalk = new Pheanstalk();
$pheanstalk->useTube('foo')->put(...);

And the worker will call watch():

$pheanstalk = new Pheanstalk();
$pheanstalk->watch('foo')->ignore('default')->reserve();

Upvotes: 3

Related Questions