CODEWITHSUNDEEP
c#azureasp.net-core.net-coreazureservicebus
ashish
ashish

Reputation: 332

Move message to 'deadletter ' in azure servicebus

I have implemented backoff exponential retry. So basically if there is any exception i clone the message and then i re-submit it to the queue by adding some delay.

Now i am facing 2 issues - 1) i see that the delivery count is not increasing when i clone and resubmit back to queue 2) I want to move it to deadletter if the max delivery count is reached.

Code :

catch (Exception ex)
                {
                    _logger.Error(ex, $"Failed to process request {requestId}");
                    var clone = messageResult.Message.Clone();
                    clone.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddSeconds(45);
                    
                    await messageResult.ResendMessage(clone);
                    if (retryCount == MaxAttempts)
                    {
                        //messageResult.dea
                    }
                    return new PdfResponse { Error = ex.ToString() };

                }

please help me on this

Upvotes: 2

Views: 4210

Answers (2)

CCRider
CCRider

Reputation: 11

We implemented a similar retry functionality over the Azure Service Bus using the "Azure.Messaging.ServiceBus" library.

The way we solved the DeliveryCount issue was to store the true delivery count value in the ApplicationProperties of the incoming message and increment it before the message is republished.

Here are parts of the code that handle this

public static ServiceBusMessage CloneMessageForRetry(this ServiceBusReceivedMessage message)
        {
            var clonedMessage = new ServiceBusMessage(message.Body) { };
            clonedMessage.CorrelationId = message.CorrelationId;
            message.ApplicationProperties.ToList().ForEach(x => 
            clonedMessage.ApplicationProperties.Add(x.Key,x.Value));

            AddOrSetOriginalMessageId(message, clonedMessage);
            AddOrSetOriginalEnqueueTime(message, clonedMessage);
            SetDeliveryCount(message, clonedMessage);

            return clonedMessage;
        }

private static void SetDeliveryCount(ServiceBusReceivedMessage originalMessage, ServiceBusMessage clonedMessage)
    {
        int currentDeliveryCount;
        if (!originalMessage.ApplicationProperties.ContainsKey(TrueDeliveryCountPropertyName))
        {
            currentDeliveryCount = originalMessage.DeliveryCount;
        }
        else
        {
            currentDeliveryCount = originalMessage.GetTrueDeliveryCount();
        }

        int trueDeliveryCount = currentDeliveryCount + 1;
        if (clonedMessage.ApplicationProperties.ContainsKey(TrueDeliveryCountPropertyName))
        {
            clonedMessage.ApplicationProperties[TrueDeliveryCountPropertyName] = trueDeliveryCount;
        }
        else
        {
            clonedMessage.ApplicationProperties.Add(TrueDeliveryCountPropertyName, trueDeliveryCount);
        }
    }

Upvotes: 0

Andreas Hütter
Andreas Hütter

Reputation: 3919

When you clone a message it becomes a new message, that means system properties are not cloned which gives the cloned message a fresh delivery count starting at 1 again. See also https://docs.azure.cn/zh-cn/dotnet/api/microsoft.azure.servicebus.message.clone?view=azure-dotnet

You can look into the Peek Lock Feature of Azure Service Bus. When using PeekLock the message gets invisible on the queue until you explicitly abandon it (put it back to the queue with delivery count increased) or complete if everything works out as expected when processing the message. Another option is to explicitly dead letter this message.

The feature is documented here: https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock

But the important thing about this is that if you do not perform any of the above mentioned actions such as cloning Azure Service Bus will automatically make the message visible again after a defined interval (the LockDuration property) or when you abandon it.

So to get a delayed retry and dead letter behaviour (when maximum delivery count has been reached) you can use the following options:

Option 1. Retry via Azure service bus auto-unlock

When processing of the message cannot be performed at the moment for some reason catch the exception and make sure none of the mentioned actions (abandon, complete or deadletter) are performed. This will keep the message invisible for the remaining time and will make it again visible after the configured lock duration has been reached. And the delivery count will also be increased by Azure Service Bus as expected.

Option 2. Implement your own retry policy

Perform your own retry policy in your code and retry processing of the message. If your maximum retries have been reached abandon the message which will make it visible again for the next queue reading step after the retry time has been reached. In this case the delivery count is increased as well.

Note: If you choose option 2.) make sure your retry period will conform to the defined LockDuration so that your message will not be visible again on the queue if you are still processing it with retries. You could also renew the lock between retries by calling the RenewLock() method on the message between retries.

If you implement the retry policy in your code I recommend using into Polly .Net which already gives you great features such as Retry and Circuit Breaker policies. See https://github.com/App-vNext/Polly

Upvotes: 2

Related Questions