CODEWITHSUNDEEP
azureazure-data-factoryazure-databricks
Shyam
Shyam

Reputation: 95

How to query Azure databricks data on Azure datafactory?

My source data is in ADLS and its table format is in ADB data.

Now I'm creating a lookup activity inside the ADF pipeline where I have to write a SELECT query on the table present in ADB.

Can you please share your suggestions on how this link can be achieved ?

Upvotes: 0

Views: 1099

Answers (2)

Himanshu Kumar Sinha
Himanshu Kumar Sinha

Reputation: 1806

if I understand the ask here you are trying to connect to an ADB instance and exeecute an query . Lookup activity does not support ADB . https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity AFAIK , you dont need to go the ADB route , you can directly query ADLS .

Upvotes: 1

joelforsyth
joelforsyth

Reputation: 1041

The query property in the lookup activity accepts an expression, not a SQL statement.

Ref: https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions

You can set the source as a Stored Procedure or you can use the Copy activity to query the results from the result of the lookup.

See the example here: https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity#pipeline

Upvotes: 1

Related Questions