CODEWITHSUNDEEP
azureazure-databricks
Yu Watanabe
Yu Watanabe

Reputation: 641

Does azure databricks support stream access fromr azure postgresql?

I have asked similar question but I would like to ask question if I can use Microsoft Azure to achieve my goal.

Is streaming input from external database (postgresql) supported in Apache Spark?

I have a database deployed on Microsoft Azure Postgresql. I have a table which I want to stream access from . Using Kafka connect , it seems that I could stream access the table, however, looking on online document , I could not find database(postgresql) as a datasource .

Does azure databricks suport stream reading postgresql table ? Or is it better to use azure HDInsight with kafka and spark ?

I appreciate if I could get some help.

Best Regards,

Yu Watanabe

Upvotes: 0

Views: 208

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12768

Unfortunately, Azure Databricks does not support stream reading of Azure postgresql database.

Azure HDInsight with Kafka and Spark will be the right choice for your requirement.

Managed Kafka and integration with other HDInsight offerings that can be used to make a complete data platform.

Azure also offers a range of other managed services needed in a data platform such as SQL Server, Postgre, Redis and Azure IoT Event Hub.

As per my research, I have found a third-party tool name "Panoply" which integrate Databricks and PostgreSQL using Panoply.

Hope this helps.

Upvotes: 1

Related Questions