CODEWITHSUNDEEP
beanstalkd
Michael
Michael

Reputation: 187

Duplicate job name in the beanstalk queue

Our system process many jobs from the queue and there are times that those jobs were not yet finish processing. There is a chance that our system will put jobs with the same name of the jobs that are currently process.

Is there a checker that will tell us that the job with the same name is already in the queue before we add it in the queue?

Thanks guys!

Upvotes: 2

Views: 285

Answers (1)

Alister Bulman
Alister Bulman

Reputation: 35169

Beanstalkd does not have the facility to look things up within it - it's a job-queue, not a giant array. Other things can be used in conjunction with it though, that do allow for random access to data to record if something has already been done.

If you know all the jobs have a specific identifier, you could put them into Redis or Memcached, probably with some form of prefix, and possibly an expiry beyond which they won't be stored.

Redis also allows for other data structures that could help as well, such as Bloom Filters and the Redis-native Hyperloglog.

Upvotes: 1

Related Questions