CODEWITHSUNDEEP
message-queuedistributed-computingzeromqbeanstalkd
Phillip B Oldham
Phillip B Oldham

Reputation: 19385

Beanstalkd to ZeroMQ: is it possible to distribute work in the same way?

A common beanstalkd workflow would be to have many workers listening for jobs on a queue/tube, locking that job while they process it, and deleting that job so that no other workers can re-process it. If the job fails (eg. resources are unavailable to complete processing) the job can slip back onto the queue for another worker to pick up the job.

Is this approach possible with ZeroMQ? Eg, using the pub/sub model can multiple subscribers receive the same job and process it at the same time? Would push/pull or req/rep provide a similar setup?

Upvotes: 1

Views: 1190

Answers (1)

harm
harm

Reputation: 10403

I'm certain ZeroMQ can provide this for you. However keep in mind that ZeroMQ is not really a queue. It's an advanced networking library. Naturally, with the provided primatives, you can do what you describe.

You specific case seems like it could be implemented as a pub/sub system, if you don't mind having the same work done many times over. I recommend reading the ZeroMQ guide and especially chapter 5.

Although I'm certain you can do what you describe with ZeroMQ, I would first search for a queue which does this already.

Upvotes: 2

Related Questions