CODEWITHSUNDEEP
c#azurequeueazure-service-fabricservice-fabric-stateful
ken2k
ken2k

Reputation: 49013

Azure Service Fabric inter service communication

I currently have one Service Fabric application that is composed of multiple Services. What I'm trying to achieve is a Queuing mechanism so one Service can publish a message to a queue, and another Service can receive messages from the same queue.

The following doesn't work (for the Listener service, there is nothing to dequeue):

PublisherService:

protected override async Task RunAsync(CancellationToken cancellationToken)
{
    var myQueue = await StateManager.GetOrAddAsync<IReliableQueue<string>>("fooQueue");
    while (true)
    {
        cancellationToken.ThrowIfCancellationRequested();
        using (var tx = this.StateManager.CreateTransaction())
        {
            // Put some message in the queue
            await myQueue.EnqueueAsync(tx, "Foobar");

            await tx.CommitAsync();
        }

        await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
    }
}

ListenerService:

protected override async Task RunAsync(CancellationToken cancellationToken)
{
    var myQueue = await StateManager.GetOrAddAsync<IReliableQueue<string>>("fooQueue");
    while (true)
    {
        cancellationToken.ThrowIfCancellationRequested();
        using (var tx = this.StateManager.CreateTransaction())
        {
            var result = await myQueue.TryDequeueAsync(tx);

            if (result.HasValue)
            {
                ServiceEventSource.Current.ServiceMessage(this.Context, "New message receieved: {0}", result.Value.ToString());
            }

            await tx.CommitAsync();
        }

        await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
    }
}

It looks like the scope of a queue is limited to a single Service. This doesn't appear to be a limitation specified in the documentation.

So my questions are:

Obviously I could use an Azure Service Bus, but I can't for several reasons:

Upvotes: 3

Views: 1876

Answers (3)

Robert
Robert

Reputation: 176

If you add a fault-handling retry pattern to all of your calling code, you should not need a queue in between your calls, see https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-communication

Relevant part from the link is here:

An exception handler is responsible for determining what action to take when an exception occurs. Exceptions are categorized into retryable and non retryable. Non retryable exceptions simply get rethrown back to the caller. retryable exceptions are further categorized into transient and non-transient. Transient exceptions are those that can simply be retried without re-resolving the service endpoint address. These will include transient network problems or service error responses other than those that indicate the service endpoint address does not exist. Non-transient exceptions are those that require the service endpoint address to be re-resolved. These include exceptions that indicate the service endpoint could not be reached, indicating the service has moved to a different node.

Upvotes: 1

Todd Abel
Todd Abel

Reputation: 439

Check out the Priority Queue Service, which was created for this purpose.

Upvotes: 1

Peter Bons
Peter Bons

Reputation: 29880

ReliableQueues are local to a service yes, because its intent is to store state for that particular service. That state is replicated to other instances. It is like a normal System.Collections.Generic.Queue<T> in .Net.

For a low cost solution maybe you can use Azure Storage Queues. Yes, it adds a dependency but it has a high availability. It is a tradeoff that only you can decide to accept or not.

On the other hand, think out of the box:

Create a stateful service with multiple ReliableQueues and expose methods other services can call using stand remoting communication like:

class QueuingService
{
    void AddToQueue<T>(string queuename, T input) { .. }
    void DeQueue(string queuename) { .. }
}

This creates of course a dependency but it has all the safety mechanisms Service Fabric provides and does not cost you much. But then again, you are building a poor mans service bus/azure storage queue yourself.

About the docs, no it does not says so with many words that a reliable queue is tied to 1 service but it depends on how you interpret this

Service Fabric offers a stateful programming model available to .NET developers via Reliable Collections. Specifically, Service Fabric provides reliable dictionary and reliable queue classes. When you use these classes, your state (my interpretation: The state of the service) is partitioned (for scalability), replicated (for availability), and transacted within a partition (for ACID semantics).

Upvotes: 3

Related Questions