CODEWITHSUNDEEP
c#azure.net-coreazureservicebus
Shahar Shokrani
Shahar Shokrani

Reputation: 8762

What is the role of "MaxAutoRenewDuration" in azure service bus?

I'm using Microsoft.Azure.ServiceBus. (doc)

I was getting an exception of:

The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue.

By the help of these questions:

1, 2, 3,

I am able to avoid the Exception by setting the AutoComplete to false and by increment the Azure's queue lock duration to its max (from 30 seconds to 5 minutes).

_queueClient.RegisterMessageHandler(ProcessMessagesAsync, new 
                         MessageHandlerOptions(ExceptionReceivedHandler)
                         {
                             MaxConcurrentCalls = 1,
                             MaxAutoRenewDuration = TimeSpan.FromSeconds(10),
                             AutoComplete = false
                         }
);

private async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
    await ProccesMessage(message);
}

private async Task ProccesMessage(Message message)
{
    //The complete should be closed before long-timed process
    await _queueClient.CompleteAsync(message.SystemProperties.LockToken);
    await DoFoo(message.Body); //some long running process
}

My questions are:

Upvotes: 18

Views: 15551

Answers (1)

Sean Feldman
Sean Feldman

Reputation: 26057

There are a few things you need to consider.

  1. Lock duration
  2. Total time since a message acquired from the broker

The lock duration is simple - for how long a single competing consumer can lease a message w/o having that message leased to any other competing consumer.

The total time is a bit tricker. Your callback ProcessMessagesAsync registered with to receive the message is not the only thing that is involved. In the code sample, you've provided, you're setting the concurrency to 1. If there's a prefetch configured (queue gets more than one message with every request for a message or several), the lock duration clock on the server starts ticking for all those messages. So if your processing is done slightly under MaxLockDuration but for the same of example, the last prefetched message was waiting to get processed too long, even if it's done within less than lock duration time, it might lose its lock and the exception will be thrown when attempting completion of that message.

This is where MaxAutoRenewDuration comes into the game. What it does is extends the message lease with the broker, "re-locking" it for the competing consumer that is currently handling the message. MaxAutoRenewDuration should be set to the "possibly maximum processing time a lease will be required". In your sample, it's set to TimeSpan.FromSeconds(10) which is extremely low. It needs to be set to be at least longer than the MaxLockDuration and adjusted to the longest period of time ProccesMessage will need to run. Taking prefetching into consideration.

To help to visualize it, think of the client-side having an in-memory queue where the messages can be stored while you perform the serial processing of the messages one by one in your handler. Lease starts the moment a message arrives from the broker to that in-memory queue. If the total time in the in-memory queue plus the processing exceeds the lock duration, the lease is lost. Your options are:

  1. Enable concurrent processing by setting MaxConcurrentCalls > 1
  2. Increase MaxLockDuration
  3. Reduce message prefetch (if you use it)
  4. Configure MaxAutoRenewDuration to renew the lock and overcome the MaxLockDuration constraint

Note about #4 - it's not a guaranteed operation. Therefore there's a chance a call to the broker will fail and message lock will not be extended. I recommend designing your solutions to work within the lock duration limit. Alternatively, persist message information so that your processing doesn't have to be constrained by the messaging.

Upvotes: 34

Related Questions