CODEWITHSUNDEEP
rabbitmqqueue
Mads Mønster
Mads Mønster

Reputation: 123

RabbitMQ lock message group

I'm seeking help from RabbitMQ or other message broker user professionals. I'm relatively new into message broker systems so please bear with me, if i'm not using the correct wording.

We have a SAAS platform where users signup, and we start to automate some processes for them. Some of these processes needs to handled on-by-one for each user. This is where we would like to use message broker.

The system is illustrated here:

enter image description here

Where we colors indicates different users. So if C-1 start to process the first task from yellow-user, C-2 must take the task from blue-user.

Suggestion 1:

enter image description here

Is it possible to "lock" a group of tasks when a consumer starts to handle a task from a specific group? In this case C-2 should then take the blue-user task. Scaling the number of consumers would be rather straightforward here.

Suggestion 2:

enter image description here

Here each user has its own queue, with one consumer attached. The challenge for us here is, how do we start a consumer automatically when a new queue is created and possibly stop the consumer if the queue i not used for a while?

Thanks in advance

Upvotes: 2

Views: 988

Answers (2)

Vanlightly
Vanlightly

Reputation: 1479

If you make the routing key the user id then you can use the consistent hash exchange to shard the messages by that key. Bind a few queues to that exchange and the messages of any given user id will exist in only a single queue. Then get c1 to subscribe to queue1, c2 to queue2 etc. You can also enable Single-Active-Consumer (SAC) on each of the queues to ensure that only a single consumer is connected to each queue at a time.

You'll need version 3.8 and onwards if you want SAC.

Upvotes: 1

José M
José M

Reputation: 3509

In order to use rabbitmq, you should use the Suggestion 2, there's no way to 'lock' messages in a queue by some key. The closest are priorities, and it does not fit your model.

Using the Suggestion 2 approach requires you to declare the queue before sending a message to it (or controlling the declaration in another way, since there's no auto-creation for queues in RabbitMQ) and using the queue TTL. You should also consider exclusive queues as an alternative, where the queue creation is controlled by the consumer instead of the publisher.

Upvotes: 0

Related Questions