Reputation: 3111
I've zipped three RDDs and as a result got a nested structure.
val rdd1 = sc.parallelize(List(1, 2, 3))
val rdd2 = sc.parallelize(List(3, 4, 5))
val rdd3 = sc.parallelize(List(6, 7, 8))
val finalRDD = rdd1.zip(rdd2).zip(rdd3)
The finalRDD
looks like
((1,3),6)
((2,4),7)
((3,5),8)
How can I flatten the structure to the following?
(1,3,6)
(2,4,7)
(3,5,8)
Upvotes: 1
Views: 150
Reputation: 336
something like this should work
finalRDD.map(r => (r._1._1,r._1._2,r._2))
Upvotes: 2