CODEWITHSUNDEEP
azure-functionsazure-eventgridazure-storage-queues
SeaDude
SeaDude

Reputation: 4405

Is there a pattern for replaying Azure Queue Storage messages?

I'm looking for a method of re-enqueueing messages if failures happen downstream after a message has been processed. Is there a pattern for this that I can study?

Example:

How can I persist the Queue Message until all downstream processes have completed?

Upvotes: 1

Views: 492

Answers (2)

PersianIronwood
PersianIronwood

Reputation: 790

Since you are using the Azure Storage Queue , the messages will have a Delivery Guarantee of At least once, which means once you receive them, they'll be hidden for visibilitytimeout duration (default to 30 seconds ) and show up in the queue afterwards. You'll have to explicitly delete them once you're done with each message, so you can postpone the deleting part after you're done with all your downstream operations. or if any of those operations fail the message will be automatically available after its visibilitytimeout finishes.

As it says in Azure documentation , using azure storage queue is one of the ways of bypassing the problem you're having :

Your application wants to track progress for processing a message in the queue. It's useful if the worker processing a message crashes. Another worker can then use that information to continue from where the prior worker left off.

Upvotes: 0

GMK
GMK

Reputation: 2950

I have a similar situation. Sometimes the message can't be processed yet because a requirement hasn't been met. I started by creating a RetryableMessage.

public class RetryableMessage<T>
{
    // number of times we try to send the message before giving up
    public int TriesRemaining = 15;
    public T Message { get; set; }
}

I put the actual message in the Message field, then, if the message processing doesn't go well, but maybe it would go well if I tried it again I do something like this

if (shouldRetry)
{
    queuedMessage.TriesRemaining--;
    if (queuedMessage.TriesRemaining > 0)
    {
        queue.Add(queuedMessage);
    }
}

That just adds it back to the end of the queue so I can try again. I included a max number of tries so if it never ends up working it stops requeuing it.

This works for my situation because it's usually processing a lot of messages and one of the others will make this one will work. If you only have one message at a time and it just doesn't work, this pattern isn't for you.

Upvotes: 1

Related Questions