CODEWITHSUNDEEP
scalaapache-spark
Peter Krauss
Peter Krauss

Reputation: 13920

Why it is an error on compiler but run at Spark-shell?

The command

println(  sql( "SELECT date_format(now(),'yyyyMMdd')" ).as[String].first )

is working fine on Spark-shell (using Spark v2.2), but the line

 val databaseNow = spark.sql( "SELECT date_format(now(),'yyyyMMdd')" ).as[String].first

not compile. I try many import variations and problem persist. I try syntax variations as df.first.toString but it is not a solution ([x] is not x).

Upvotes: 0

Views: 103

Answers (1)

Emiliano Martinez
Emiliano Martinez

Reputation: 4123

Just include the included encoders like :

val spark: SparkSession =
  SparkSession
    .builder()
    .appName("test")
    .getOrCreate()

import spark.implicits._

val databaseNow = spark.sql( "SELECT date_format(now(),'yyyyMMdd')" ).as[String].first

Upvotes: 2

Related Questions