CODEWITHSUNDEEP
c#azureservicebusmicroservices
shlatchz
shlatchz

Reputation: 1652

Lock a Service-Bus Queue and prevent others from accessing it

I have multiple queues that multiple clients insert messages into them.

On the server side, I have multiple micro-services that access the queues and handle those messages. I want to lock a queue whenever a service is working on it, so that other services won't be able to work on that queue.

Meaning that if service A is processing a message from queue X, no other service can process a message from that queue, until service A has finished processing the message. Other services can process messages from any queue other than X.

Does anyone has an idea on how to lock a queue and prevent others from accessing it? preferably the other services will receive an exception or something so that they'll try again on a different queue.

UPDATE

Another way can be to assign the queues to the services, and whenever a service is working on a queue no other service should be assigned to the queue, until the work item was processed. This is also something that isn't easy to achieve.

Upvotes: 4

Views: 2090

Answers (5)

BeshEater
BeshEater

Reputation: 152

You can achieve this by using Azure Service Bus message sessions

All messages in your queue must be tagged with the same SessionId. In that case, when a client receives a message, it locks not only this message but all messages with the same SessionId (effectively whole queue).

Upvotes: 2

Dax Fohl
Dax Fohl

Reputation: 10781

There are several built-in ways of doing this. If you only have a single worker, you can set MessageOptions.MaxConcurrentCalls = 1.

If you have multiple, you can use the Singleton attribute. This gives you the option of setting it in Listener mode or Function mode. The former gives the behavior you're asking for, a serially-processed FIFO queue. The latter lets you lock more granularly, so you can specifically lock around critical sections, ensuring consistency while allowing greater throughput, but doesn't necessarily preserve order.

My Guess is they'd have implemented the singleton attribute similarly to your Redis approach, so performance should be equivalent. I've done no testing with that though.

Upvotes: 2

shlatchz
shlatchz

Reputation: 1652

The solution was to use Azure's redis to store the locks in-memory and have micro-services that manage those locks using the redis store.

The lock() and unlock() operations are atomic and the lock has a TTL, so that a queue won't be locked indefinitely.

Upvotes: 1

usr
usr

Reputation: 171178

Put the work items into a relational database. You can still use queues to push work to workers but the queue items can now be empty. When a worker receives an item he know to look into the database instead. The content of the message is disregarded.

That way messages are independent and idempotent. For queueing to work these two properties usually must hold.

That way you can more easily sequence actions that actually are sequential. You can use transactions as well.

Maybe you don't need queues at all. Maybe it is enough to have a fixed number of workers polling the database for work. This loses auto-scaling with queues, though.

Upvotes: 0

Sean Feldman
Sean Feldman

Reputation: 25994

Azure Service Bus is a broker with competing consumers. You can't have what you're asking with a general queue all instances of your service are using.

Upvotes: 0

Related Questions