CODEWITHSUNDEEP
scalaapache-spark
PKM15
PKM15

Reputation: 109

Conversion of RDD to Dataframe

I read a csv file to RDD and trying to convert it to DataFrame. But, it throughs error.

scala> rows.toDF()
<console>:34: error: value toDF is not a member of org.apache.spark.rdd.RDD[Array[String]]
              rows.toDF()

scala> rows.take(2)       
    Array[Array[String]] = Array(Array(1, 0, 3, "Braund, ...

What am I doing wrong?

Upvotes: 2

Views: 1350

Answers (1)

eliasah
eliasah

Reputation: 40360

When you want to convert an RDD to a DataFrame, you'll need to create an SQLContext and import it's implicit functions like @zero323 suggested. 

import sqlContext.implicits._
rows.toDF

In case your RDD is a RDD[Row] , the following will be needed 

import org.apache.spark.sql.Row
rows.map(Row.fromSeq(_)).toDF

Upvotes: 3

Related Questions