Ruby Resque requeueing when similar job is already running?

Question

So I noticed that resque-lock is about preventing jobs to be queued. But what if I want my queue to be filled entirely, but I want to requeue when a specific situation is locked. To clear that up: Because my queue consists of a bunch of hosts (for ex. google.com, yahoo.com, yahoo.com and apple.com), I don't want two of the same hosts entering different workers at the same time! How do I do that? I got resque-lock working but I don't think it'll help with this problem.

Example:

Queue:

google.com
yahoo.com
apple.com
apple.com
stackoverflow.com

Workers:

• 2 workers are looking for jobs and find: google.com and yahoo.com (okay)
• 2 workers are looking for new jobs and find: apple.com and apple.com
• Not okay, put the second apple.com job back up in the queue and continue

Any ideas on how to get this behaviour with resque? My example described the situation with 2 workers but the main idea is just that two jobs with the same host should not be able to run at the same time, regardless of the amount of workers. Jobs that are not allowed to run should be put back up in the queue.

Answer 1

I made my own resque plugin for this, called resque::workers::lock.