Can I access ADLS from Azure Databricks using an ADF managed identity?

Question

I have a ADF which has a managed identity able to access the ADLS. Now I need this ADF to call a Databricks notebook and this notebook should access the ADLS as well. Can I do it using the managed identity from ADF? How can I do this?

Answer 1

Right now the Azure Active Directory credentials passthrough doesn't work with service principals & managed identity. You can use managed identity to connect to the Databricks workspace itself (see docs), but from the workspace you need to setup something to access data on ADLS. You have two possibilities:

• mount the ADLS into Databricks workspace using the service principal
• setup necessary Spark properties to connect to ADLS directly using the service principal