Sang
Reputation: 31
Scala RDD[(String,String)] to RDD[String]
We are reading a file in UTF_8 as RDD[(String,String)], Need help in converting this to RDD[String].
val textRdd = sparkSession.sparkContext.binaryFiles(filePath,12)
.mapValues(content => new String(content.toArray(), StandardCharsets.UTF_8))
Upvotes: 0
Views: 62
Answers (1)
Mohana B C
Reputation: 5487
You just need to extract value from tuple i.e (string, string).
val outRDD = textRdd.map(t=>t._2) // To get first value use t._1
Upvotes: 1
Related Questions
- Convert RDD[String] to Data Frame
- Flatten RDD[(String,Map[String,Int])] to RDD[String,String,Int]
- How to convert RDD[Row] to RDD[String]
- Scala RDD[String] to RDD[String,String]
- Convert Array[(String,String)] type to RDD[(String,String)] type in spark
- How to convert RDD[(String, String)] into RDD[Array[String]]?
- How to convert a RDD<String> to a RDD<Vector> in Spark?
- Spark RDD[(String, String)] into RDD[Map[String, String]]
- How should I do to transform a RDD[String] to RDD[(String, String)]?
- convert scala string to RDD[seq[string]]