CODEWITHSUNDEEP
phpqueuejobsgearman
Nands
Nands

Reputation: 1531

Gearman: Restrict workers from picking jobs based on some input params

I've a Gearman queue which processes some user specific data via multiple workers. I do not want a particular user to occupy more than a single worker at once.

Say I've a queue named process_user_data() and I run 4 workers W1, W2, W3, W4 When Userid 1 submits 10 jobs, I want only W1 to process it. W2-W4 should not pick the jobs.

Is this do able in gearman?

Upvotes: 2

Views: 1135

Answers (2)

Aurimas
Aurimas

Reputation: 2493

No Gearman does not support this natively. I believe the easiest way would be to prefix / suffix the functions to indicate the user they belong to. E.g: user 1 job should be submitted to process_data_1(), and worker 1 would hook up on that instead of a generic process_data(). Internally, the workers could still have the same code base, as it would be only a matter of the hook to the Gearman server (which you could manage when launching the worker via a command line parameter):

class Worker

public function __construct() {
 $this->user = argv[1];
 $this->worker = new GearmanWorker();
 $this->worker->addServer();
 $this->worker->addFunction("process_data_" . $this->user, array($this, 'process_data'));
}

public function process_data() {
 //work code
}
}

Upvotes: 1

halfer
halfer

Reputation: 20439

It's easy if the workers are on different servers - just don't declare them with GearmanClient->addServer() in your client. If there're on the same server, I suppose you could have several instances of gearmand running on a single machine, on a range of ports. You could then use addServer(host, port) to selectively declare which 'groups' should stand a chance of being offered the job/task you're just about to add.

If that's too messy for you, then you may need to choose a different queueing system. I seem to remember reading the Gearman doesn't allow for job-type based filtering, though I'm afraid I don't have a reference for that. Maybe look at RabbitMQ or BeanStalk?

Upvotes: 0

Related Questions