CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
lahsrah
lahsrah

Reputation: 21

not found: value udf error

I have a udf defined in my code as follows:

val toOriginalTimestamp = udf((timestamp: String) => timestamp.substring(0, 18))

I'm trying to take a substring of the timestamp field in my dataset. However I'm getting the error as not found: value udf

What am I doing wrong?

Upvotes: 2

Views: 3843

Answers (1)

TheMP
TheMP

Reputation: 8427

You probably need this one:

import org.apache.spark.sql.functions.udf

Also, make sure that you are using spark-sql (not just spark-core!) 1.3 or higher dependencies.

Upvotes: 11

Related Questions