CODEWITHSUNDEEP
spark-streamingazure-data-explorer
Dhiraj
Dhiraj

Reputation: 3696

Can we stream data from ADX to Databricks Spark cluster?

Does ADX support sending data to Databricks Spark cluster in streaming fashion -- basically Spark will pull it from ADX instead of ADX exporting it? In other words, I am trying to understand if an ADX table can act as a source for spark streaming ? Is there an example link for this that I can go through?

Upvotes: 0

Views: 469

Answers (1)

Ziv Caspi
Ziv Caspi

Reputation: 76

ADX provides the means for querying a table for all the data that has been added to it since the last query through the means of Database Cursors. Note, however, that this requires the caller to parse the query's @ExtendedProperties set (which holds the database cursor) and maintain state between every two successive queries (so that the new cursor value could be passed to the next query).

Upvotes: 2

Related Questions