CODEWITHSUNDEEP
scalaapache-spark
ashwinsakthi
ashwinsakthi

Reputation: 1956

Why import implicit SqlContext.implicits._ after initializing SQLContext in a scala spark application

Why import implicit SqlContext.implicits._ after initializing SQLContext in a scala spark application.

when import is put outside the object , there seems to be an issue.

I am from a java back ground , where am not understanding the usage of import statements within a def object.

val sqlContext = new SQLContext(sc)

import sqlContext.implicits._
import sqlContext._

Upvotes: 4

Views: 1695

Answers (1)

Yuval Itzchakov
Yuval Itzchakov

Reputation: 149538

The import doesn't work externally because the implicits object is defined inside the SQLContext class:

* :: Experimental ::
* (Scala-specific) Implicit methods available in Scala for converting
* common Scala objects into `DataFrame`s.
*
* {{{
*   val sqlContext = new SQLContext(sc)
*   import sqlContext.implicits._
* }}}
@Experimental
@InterfaceStability.Evolving
object implicits extends SQLImplicits with Serializable {
  protected override def _sqlContext: SQLContext = self
}

Thus, only once you have an instance of SQLContext can you import the internal object definition containing all the implicits.

Upvotes: 5

Related Questions