CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
Db8
Db8

Reputation: 93

java.lang.ClassCast errors.GenericRowWithSchema cannot be cast to scala.collection.Seq

How to convert the RDD of map to wrapped Array, I am getting error

Schema:

enter image description here

When I am trying to convert dataframe to pojo, I am getting exception as below:

java.lang.ClassCastException: 
  org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot 
  be cast to scala.collection.Seq

Code:

rdd.map(row => {
  var arrm_list: Seq[Row] = rows.getAs[AnyRef]("ArrTeber").asInstanceOf[Seq[Row]]
  //working fine here
  arrm_list.foreach(x => {
    var arrmtrb__list: Seq[Row] = rows.getAs[AnyRef]("ArrTeberPRD").asInstanceOf[Seq[Row]]

    //working fine here
    arrmtrb__list.foreach(pt => {

      var pd__list: Seq[Row] = rows.getAs[AnyRef]("Pduct").asInstanceOf[Seq[Row]] //raising error
    })
  })
})

Upvotes: 0

Views: 10852

Answers (1)

hagarwal
hagarwal

Reputation: 1163

The above exception is simply a class cast exception, Since struct cannot be cast to Seq of struct (Refer Schema: -- Pduct: struct (nullable = true) ). Cast Pduct to Row and then extract nested members.

df.map(row => {
  var arrm_list: Seq[Row] = row.getAs[Seq[Row]]("ArrTeber")

  arrm_list.foreach(x => {
    var arrmtrb__list: Seq[Row] = x.getAs[Seq[Row]]("ArrTeberPRD")


    arrmtrb__list.foreach(pt => {
      var pd__list: Row = pt.getAs[Row]("Pduct") //Pduct: struct (nullable = true)

    })
  })

})

Upvotes: 4

Related Questions