CODEWITHSUNDEEP
scalaapache-sparkkryo
Will
Will

Reputation: 2227

Spark Kryo registration throws IllegalArgumentException

When I am running my program, Spark pops this error

java.lang.IllegalArgumentException: Class is not registered: com.domain.package.program.model.ParameterEvenementData[] 
Note: To register this class use: kryo.register(com.domain.package.program.model.ParameterEvenementData[].class);

I am doing exactly what is said in the documentation and still, this error pops

I am using Scala 2.10 and spark 1.3 and here's my registration conf

object SparkUtil extends Serializable {

  def initialize = {

    val conf = new SparkConf()
    .set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
    .set("spark.kryo.registrationRequired","true")
    .set("spark.kryoserializer.buffer.mb","24") 
    .setAppName("com.domain.package.program")

    conf.registerKryoClasses(Array(classOf[ParameterEvenementData], classOf[ClientData], classOf[EsdAdresseData]))

    val sc = new SparkContext(conf)

    val sqlc = new SQLContext(sc)
    sqlc
  }

Am I doing something wrong here ? Any help will be appreciated

Upvotes: 0

Views: 89

Answers (0)

Related Questions