CODEWITHSUNDEEP
asp.net-coreazureservicebusazure-servicebus-topics
Umair
Umair

Reputation: 3253

Azure service bus multiple instances for the same subscriber

I have a situation where I have an asp.net core application which registers a subscription client to a topic on startup (IHostedService), this subscription client essentially has a dictionary of callbacks that need to be fired whenever it detects a new message in a topic with an id (this id is stored on the message properties). This dictionary lives throughout the lifetime of the application, and is in memory.

Everything works fine on a single instance of the asp.net core app service on azure, as soon as I scale up to 2, I notice that sometimes the callbacks in the subscription are not firing. This makes sense, as we have two instances now, each with its own dictionary store of callbacks.

So I updated the code to check if the id of the subscription exists, if not, abandon message, if yes, get the callback and invoke it.

public async Task HandleMessage(Microsoft.Azure.ServiceBus.Message message, CancellationToken cancellationToken)
{
    var queueItem = this.converter.DeserializeItem(message);

    var sessionId = // get the session id from the message
    if (string.IsNullOrEmpty(sessionId))
    {
         await this.subscriptionClient.AbandonAsync(message.SystemProperties.LockToken);
         return;
    }

    if (!this.subscriptions.TryGetValue(sessionId, out var subscription))
    {
        await this.subscriptionClient.AbandonAsync(message.SystemProperties.LockToken);
        return;
    }

    await subscription.Call(queueItem);

    // subscription was found and executed. Complete message
    await this.subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);
}

However, the problem still occurs. My only guess is that when calling AbandonAsync, the same instance is picking up the message again?

I guess what I am really trying to ask is, if I have multiple instances of a topic subscription client all pointing to the same subscriber for the topic, is it possible for all the instances to get a copy of the message? Or is that not guaranteed.

Upvotes: 3

Views: 5988

Answers (2)

Umair
Umair

Reputation: 3253

I managed to solve the issue by making use of service bus sessions. What I was trying to do with the dictionary of callbacks is basically a session manager anyway!

Service bus sessions allow me to have multiple instances of a session client all pointing to the same subscription. However, each instance will only know or care about the sessions it is currently dealing with.

Upvotes: 2

Sean Feldman
Sean Feldman

Reputation: 26057

if I have multiple instances of a topic subscription client all pointing to the same subscriber for the topic, is it possible for all the instances to get a copy of the message? Or is that not guaranteed.

No. If it's the same subscription all clients are pointing to, only one will be receiving that message.

You're running into an issue of scaling out with competing consumers. If you're scaling out, you never know what instance will pick the message. And since your state is local (in memory of each instance), this will fail from time to time. Additional downside is the cost. By fetching messages on the "wrong" instance and abandoning, you're going to pay higher cost on the messaging side.

To address this issue you either need to have a shared/centralized or change your architecture around this.

Upvotes: 6

Related Questions