CODEWITHSUNDEEP
javascalaapache-spark
TakeSoUp
TakeSoUp

Reputation: 8057

Spark converting a Dataset to RDD

I have a Dataset[String] and need to convert to a RDD[String]. How?

Note: I've recently migrated from spark 1.6 to spark 2.0. Some of my clients were expecting RDD but now Spark gives me Dataset.

Upvotes: 14

Views: 22071

Answers (2)

cheseaux
cheseaux

Reputation: 5315

As stated in the scala API documentation you can call .rdd on your Dataset :

val myRdd : RDD[String] = ds.rdd

Upvotes: 28

user3215496
user3215496

Reputation: 29

Dataset is a strong typed Dataframe, so both Dataset and Dataframe could use .rdd to convert to a RDD.

Upvotes: 1

Related Questions