CODEWITHSUNDEEP
azureazureservicebusazure-servicebus-queuesorchestration
Mahesh Gupta
Mahesh Gupta

Reputation: 487

how to Orchestrate an event from azure service bus queue to multiple topics?

in my application I have a scenario where there are events are continuously generated, each event is associated with a type.

For every event multiple sub actions depending on the type need to be performed which are totally independent.

So for this scenario solution which I thought of is to use Queue along with topics as below

Event generator ---> Queue -----> Orchestrator --------> topics <----- Listeners

to implement the above there is a need to have an orchestrator which gets the events from the queue and apply some logic and insert into topics .

Can any one suggest any good component for the orchestrator on azure ?

Orchestrator should be able to handle load(Spikes are expected) - As we are using queues processing an event need not be a realtime it can be a delayed based on the load.

Upvotes: 0

Views: 511

Answers (1)

Mikhail Shilkov
Mikhail Shilkov

Reputation: 35144

I have two options for you.

I. Service Bus forwarding

If the rules of forwarding are straightforward (e.g. based on a property value), you can setup this with Service Bus filterer subscriptions and forwarding.

You create an ingress topic which will get all the messages. Then you create a subscription for this topic per each type of message, and you apply a filter for that subscription to filter out all the other types. You then setup the forward rule to send the messages to the appropriate outgoing topic:

var description = new SubscriptionDescription("IngressTopic", "Type1");
description.ForwardTo = "SinkType1Topic";
SqlFilter filter = new SqlFilter("Type = 1");
namespaceManager.CreateSubscription(description, filter);

where Type is the property in your message. You repeat this for each type.

II. Azure Function

If you need to run your custom code to determine which outgoing topic the message belongs to, you can run this custom code as Azure Function. Just make an ingress queue, and create a Function which will listen to this queue, receive the message and forward it to the appropriate outgoing topic:

public static void Run(MyMessage message, IBinder binder)
{
    string outgoingTopic = message.CalculateOutgoingTopic();
    var attribute = new ServiceBusAttribute(outgoingTopic);
    var collector = binder.Bind<ICollector<MyMessage>>(attribute);
    collector.Add(message);
}

Azure Functions are good for handling spiky load.

Upvotes: 1

Related Questions