CODEWITHSUNDEEP
azurepublish-subscribeazureservicebusazure-servicebus-topics
chugh97
chugh97

Reputation: 9992

What is the best approach to copy messages from one service bus to another?

Let me give you a bit of background. I have an IoT app which is streaming data to a Service bus which has Topic filters with multiple subscribers on the LIVE environment, everything is good. Now I'm trying to create a test environment which is completely separate. As the data is streaming into LIVE I would ideally like a copy of all the messages arriving into this Test environment as well, so we can isolate the environments for the test team. So far the solutions proposed have been, is to add a second service bus connection in code and add the messages to both the buses the live one and the test one. This requires code change in many areas of the app and is clunky. I am looking for a more elegant approach where I can copy the messages arriving on a second bus. The first bus gets a copy and the second bus also gets a copy. Any advice would be appreciated?

Upvotes: 1

Views: 4144

Answers (1)

Sean Feldman
Sean Feldman

Reputation: 26012

Azure Service Bus does not support cross namespace forwarding as of today. If that would be possible, you'd be able to set up a subscription with auto-forwarding to another namespace. Until then, you will need to set up something custom indeed.

So far the solutions proposed have been, is to add a second service bus connection in code and add the messages to both the buses the live one and the test one. This requires code change in many areas of the app and is clunky.

In addition to that, it introduces the testing concern into your production application, which doesn't feel right. An approach I would try (note there are other options that could work as well) would be to have an additional subscription entity and introduce an Azure Function triggered by ServiceBusTrigger configured to the subscription entity you'll set up. The function would be able to leverage a ServiceBus output binding configured to use the testing namespace. The benefits of this approach are:

  1. Your production application doesn't have to change.
  2. You have complete control over what messages are pushed to the testing namespace vias filtering on the subscription.
  3. You have control over flow of messages into the function by disabling/enabling the subscription or deleting it along with the function.

You will incur some additional cost for Function execution.

Pseudo code:

[FunctionName("CloneForTesting")]
[return: ServiceBus(TopicName = "topic", SubscriptionName = "subscription", Connection = "ProductionServiceBusConnection")]
public static string ServiceBusOutput([ServiceBusTrigger(QueueName = "queue", Connection = "TestingServiceBusConnection")] 
Message message, ILogger log)
{
    log.LogInformation($"Cloning message with ID {message.MessageId}");
    return message.Clone();
}

Upvotes: 4

Related Questions