CODEWITHSUNDEEP
azuredatabricksazure-databricks
Dicer
Dicer

Reputation: 75

Where are the Azure Databricks DBFS datasets stored?

I upload data to the Azure Databricks and I can see there are datasets in the DBFS list, but I cannot find any datasets in the resource group that Databricks belongs to.

Where are Azure Databricks DBFS datasets stored?

Also, if I create Delta Lake, how can I call the delta lake from Azure Data factory? I cannot see where the delta lake is stored.

Upvotes: 1

Views: 6628

Answers (1)

Vamsi Bitra
Vamsi Bitra

Reputation: 2729

Where are Azure Databricks DBFS datasets stored?

The Default storage location in DBFS is known as the DBFS root. You can find any datasets in /databricks-datasets: See special DBFS Root location.

Ref1

Databricks File System (DBFS) is a distributed file system mounted into an Azure Databricks workspace and available on Azure Databricks clusters. DBFS is on top of scalable object storage ADLS gen2.

how can I call the delta lake from Azure Data factory?

You can refer to this video and also Follow below References :

https://www.youtube.com/watch?v=evqQRwsF_Ps

Reference:

https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-databricks-delta-lake?tabs=data-factory

https://learn.microsoft.com/en-us/azure/databricks/data/databricks-file-system#--important-information-about-dbfs-permissions

https://learn.microsoft.com/en-us/azure/databricks/data/databricks-datasets

Upvotes: 6

Related Questions