CODEWITHSUNDEEP
azurepysparkdatabricksmount-point
Nuno Rebelo
Nuno Rebelo

Reputation: 1

Azure Databricks dbfs with python

In azure databricks i have different results for the directory list of dbfs by simply adding two dots. Can anybody explain to me why this happens?

case one

case two

Upvotes: 0

Views: 462

Answers (1)

Axel R.
Axel R.

Reputation: 1300

With dbutils, you can only use "dbfs:/" paths. If you do not specify "dbfs:/" at the start of your path, it will simply auto-add it.

dbutils.fs.ls('pathA')
--> dbfs:/pathA

is exactly the same as

dbutils.fs.ls('dbfs:/pathA')

but if you do not use the ':', then it will add it silently.

dbutils.fs.ls('dbfs/pathB')
--> dbfs:/dbfs/pathB

It means your dbfs/ is considered as a folder name dbfs at the root of your dbfs:/

To avoid confusion, always specify dbfs:/ to your path.

Upvotes: 2

Related Questions