CODEWITHSUNDEEP
scalaapache-spark
Delip
Delip

Reputation: 465

Flattening a Scala Map in an RDD

I have an an RDD that looks like:

uidProcessedKeywords: org.apache.spark.rdd.RDD[(Long, Map[String,Double])]

How do I flatten the map in the RDD to get this:

org.apache.spark.rdd.RDD[(Long, String, Double)]

Upvotes: 3

Views: 962

Answers (1)

Nikita
Nikita

Reputation: 4515

val x = sc.parallelize(List((2, Map("a" -> 0.2, "b" -> 0.3))))
x.flatMap { 
    case (id, m) => m.map { case (k, v) => (id, k, v)}
  }
 .collect()
res1: Array[(Int, String, Double)] = Array((2,a,0.2), (2,b,0.3))

Upvotes: 7

Related Questions