CODEWITHSUNDEEP
scalaapache-sparkapache-zeppelin
Daebarkee
Daebarkee

Reputation: 683

scala object Task not serializable

object Util extends Serializable {
    def add(str: String): String = str + " ha ha"
}

object Job  extends Serializable {
    def start(days: Int): Unit = {
        import spark.implicits._
        val data = (0 to days).toList.toDS.map(x => Util.add(x.toString))

        data.collect.foreach(println)
    }
}
Job.start(10)

I made a very simple script to test why Task-Not-Serializable issue happens in Zeppelin. If I change Util.add(x.toString) to x.toString + " ha ha", then, totally fine. But I wonder why I cannot use the object Util here.

Upvotes: 0

Views: 79

Answers (1)

Daebarkee
Daebarkee

Reputation: 683

This was due to Zeppelin. I tested the same thing on the other spark shell and it worked. I hope this case helps the other people's time.

Upvotes: 1

Related Questions