CODEWITHSUNDEEP
databricksazure-databricks
czajek
czajek

Reputation: 699

Using service principal to access blob storage from Databricks

I followed Access an Azure Data Lake Storage Gen2 account directly with OAuth 2.0 using the Service Principal and want to achieve the same but with blob storage general purpose v2 (with hierarchical fs disabled). Is it possible to get this working, or authenticating using access key or SAS is the only way?

Upvotes: 2

Views: 2382

Answers (2)

Tomasz Romaniszyn
Tomasz Romaniszyn

Reputation: 1

Blob Storage using OAuth 2.0 with an Azure service principal will work only for ADLS Gen2 with Hierarchical namespace enable. It looks like even if your storage shows ADLSv2 but Hierarchical namespace is disable it will not allow for ABFS with SP.

If you use normal Azure storage ( without Hierarchical namespace enable) only WASB should be possible but Service Principal doesn’t work- only supporting token based access.

Upvotes: 0

Hauke Mallow
Hauke Mallow

Reputation: 3182

No that is not possible as of now. OAuth Bearer Token is supported for Azure Data Lake Storage Gen2 (with the hierarchical namespace enabled when creating the storage account). To access Azure Data Lake Store Gen2 the ABFS-driver is used:

abfss://<your-file-system-name>@<your-storage-account-name>.dfs.core.windows.net/

To access the Blob Storage you use WASB:

wasbs://<your-container-name>@<your-storage-account-name>.blob.core.windows.net

only supporting token based access.

Upvotes: 4

Related Questions