CODEWITHSUNDEEP
apache-sparkdatabricksdatabricks-connect
Srinivas
Srinivas

Reputation: 2100

Spark implicits do not work on dbutils object

Why cannot I do somthing like this?

import spark.implicits._
dbutils.fs.ls(folderName).toDF

throws an error of

value toDF is not a member of Seq[com.databricks.backend.daemon.dbutils.FileInfo]

But it works fine in databricks notebook. Can anyone advice what I am missing?

I am using the following library "com.databricks" %% "dbutils-api" % "0.0.4"

Upvotes: 0

Views: 374

Answers (1)

Sparker0i
Sparker0i

Reputation: 1851

From the relevant Databricks page:

DBUtils are not supported outside of notebooks

Upvotes: 1

Related Questions