CODEWITHSUNDEEP
pythonazuredatabricksazure-databricks
pelos
pelos

Reputation: 1876

databricks with python can't use fs module AttributeError: module 'dbutils' has no attribute 'fs'

I am using azure databricks for the first time to read some files and trying to use python with dbutils.fs.ls("/mnt")

But I get an error saying that dbutils doesn't have fs module. I was reading and say that all databricks come with dbutils already.

AttributeError: module 'dbutils' has no attribute 'fs'

enter image description here

if i do 

print(dir(dbutils))

['Console', 'DBUtils', 'FileInfo', 'Iterable', 'ListConverter', 'MapConverter', 'MountInfo', 'NotebookExit', 'Py4JJavaError', 'SecretMetadata', 'SecretScope', 'WidgetsHandlerImpl', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'spec', 'absolute_import', 'makeTensorboardManager', 'namedtuple', 'print_function', 'range', 'stderr', 'stdout', 'string_types', 'sys', 'zip']

i found that it suppose to have the library already install https://docs.databricks.com/user-guide/dev-tools/dbutils.html#dbutils

Is there a magic trick? I wanted to check if I have a file is mounted if not mount it and unmount it.

Upvotes: 4

Views: 15276

Answers (2)

Paul H
Paul H

Reputation: 101

You will get this error "AttributeError: module 'dbutils' has no attribute 'fs'" if you import dbutils in your notebook. There is no need to do this import as the dbutils are available in the notebook on startup. print(dir(dbutils)) should return

['CredentialsHandler', 'FSHandler', 'LibraryHandler', 'NotebookHandler', 'PreviewHandler', 'SecretsHandler', 'call', 'class', 'delattr', 'dict', 'dir', 'doc', 'eq', 'format', 'ge', 'getattr', 'getattribute', 'getstate', 'gt', 'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', 'entry_point', 'help', 'shell', 'widgets']

You don't need to restart the cluster, you can 'Clear State' from the clear tab in your workspace, or you can 'import pyspark.dbutils'.

Upvotes: 10

Vijay Kumar Sharma
Vijay Kumar Sharma

Reputation: 397

The following command dbutils.fs.ls("/mnt") is working fine in databricks, if the issue persists continue, please restart your cluster.

for the reference visit the following link

Upvotes: 0

Related Questions