CODEWITHSUNDEEP
scalaapache-sparkapache-spark-2.0
Max van Daalen
Max van Daalen

Reputation: 327

Scala case class ignoring import in the Spark shell

I hope there is an obvious answer to this question!

I've just upgraded to Spark v2.0 and have an odd problem with the spark-shell (Scala 2.11 build).

If I enter the following minimal Scala,

import java.sql.Timestamp

case class Crime(caseNumber: String, date: Timestamp, description: String, detail: String, arrest: Boolean)

I get the following error,

<console>:11: error: not found: type Timestamp

If I use the Java Timestamp class elsewhere, e.g. in a function, then no errors are generated (as you would expect because of the import).

If I fully qualify and use java.sql.Timestamp in the case class it works!

Am I missing something obvious?

Upvotes: 8

Views: 8097

Answers (1)

Mikel San Vicente
Mikel San Vicente

Reputation: 3863

It's just that the Timestamp is not loaded in the case class declaration, to fix this you can:

:paste
import java.sql.Timestamp
case class Crime(caseNumber: String, date: Timestamp, description: String, detail: String, arrest: Boolean)

or

case class Crime(caseNumber: String, date: java.sql.Timestamp, description: String, detail: String, arrest: Boolean)

Upvotes: 12

Related Questions