CODEWITHSUNDEEP
pythonazuregoogle-cloud-platformgoogle-cloud-dataflowapache-beam
Harvey
Harvey

Reputation: 786

Dataflow to Read from Azure Service Bus

I want to know if I can connect to a Azure Bus Queue using Dataflow directly.

Information is scarce on this topic so I have turned here to see if anyone has experience trying to make such a connection.

The overall idea is to have a Apache Beam Pipeline running on google dataflow that will read from Azure Service Bus and then write the data to Google Big Query.

Upvotes: 0

Views: 528

Answers (1)

Mdumanoj
Mdumanoj

Reputation: 527

There is no IO provided by Apache Beam to connect with Azure Service Bus. There is only one IO available for Azure cloud to connect with Azure blob storage.

Apparently you can achieve the same thing using Azure Event Hub service. It has a Kafka protocol on top of it so you can use KafkaIO to connect with Event Hub.

Upvotes: 1

Related Questions