CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sqltuples
Sood
Sood

Reputation: 149

Convert a RDD[(Long,Long)] to RDD[Row]

How do I convert a RDD[(Long, Long)] to RDD[Row]?

I require the RDD[Row] to convert to a DataFrame. The closest I got to an answer is How to convert RDD[Row] to RDD[Vector] but I want to do the opposite.

Upvotes: 2

Views: 719

Answers (1)

banjara
banjara

Reputation: 3890

Just apply schema to your RDD (ref: spark-sql-programming-guide)

case class MyObjectType(col1:Long, col2:Long)

val myRDD:RDD[Long, Long] = .........
val myDF = myRDD.map(r=>MyObjectType(r._1, r._2)).toDF

now if you want to run sql over this df, you can register it as temp table

myDF.createOrReplaceTempView("my_table")

Upvotes: 4

Related Questions