Azure Service Bus Queues - How to prevent IHostedService from processing queue messages immediately

Question

Background - I'm creating and placing messages on an Azure Service Bus queue and processing the messages using IHostedService.

Question - Is there any way I can create a delay in my IHostedService MessageHandler method to not process messages immediately after they have been placed on the queue?

The reason I want to do this is that after I place a message on the queue I want to take the identifier returned and store it in the database along with a few other steps.But at the moment my message is picked up and processed by the IHostedService immediately

Answer 1

You could also add an intermediate step with another queue. So you could put the message in queueA (new queue instead of your existing one) and have an Azure function or an IHostedService reading from that queue and writing the data in the db and anything else you want to do. Then you can put the same message or a new one with more information on queueB (the queue you already have) and process it without changing anything in your existing code.

If you can use Azure, check Logic Apps. They would fit nicely to what you try to achieve.

Answer 2

Messaging is about decoupling producers and consumers. This means the consumer should not know anything about the constraints of the producer and vice versa. Messaging is decoupling the two parties. Which means whenever there’s a message in a queue, the processor should be able to process it.

The suggested option of scheduling a message with a delay would work but it ha has a potential flaw. If a message is scheduled and its sequence number has been obtained but the system is failing to access the database right at that moment, there’s a good probability for the message to show up on the queue but the database won’t have the sequence number.

What are the alternatives?

Message deferral

Deferred messages remain in the main queue along with all other active messages (unlike dead-letter messages that live in a subqueue), but they can no longer be received using the regular receive operations.

To retrieve a deferred message, its owner is responsible for remembering the sequence number as it defers it. Any receiver that knows the sequence number of a deferred message can later receive the message by using receive methods that take the sequence number as a parameter.

source

What the consumer, IHostedService then could do is receive a message, check if the additional information in the database is available, and if not, defer the message accompanied with a scheduled message to itself that would contain the deferred message’s sequence number, by which the original (deferred) message could be retrieved. This way you can control number of attempts and decide on what to do if the accompanying steps were never written to the database.

Routing slip pattern

The reason I want to do this is that after I place a message on the queue I want to take the identifier returned and store it in the database along with a few other steps.

If the steps are related to the subsequent processing such as the next destination the message should be sent to, or similar, the routing slip pattern is more appropriate. You would package the other steps/additional data along with the message and ship it. That way the message is self sustained, not relying on the database, doesn’t have to be scheduled/delayed/deferred, and at every step could be augmented and modified depending on the flow.

Answer 3

You can schedule the message for a later time. See for example: https://medium.com/event-driven-utopia/azure-service-bus-essentials-scheduled-messages-98fd542f9d55.

You can set the ScheduledEnqueueTimeUtc property to a UTC DateTime in the future and it will only become visible to the consumer at that time.