samba
samba

Reputation: 3111

How to flatten multiple zipped RDDs?

I've zipped three RDDs and as a result got a nested structure.

val rdd1 = sc.parallelize(List(1, 2, 3))
val rdd2 = sc.parallelize(List(3, 4, 5))
val rdd3 = sc.parallelize(List(6, 7, 8))

val finalRDD = rdd1.zip(rdd2).zip(rdd3)

The finalRDD looks like

((1,3),6)
((2,4),7)
((3,5),8)

How can I flatten the structure to the following?

(1,3,6)
(2,4,7)
(3,5,8)

Upvotes: 1

Views: 150

Answers (1)

Ilya Brodezki
Ilya Brodezki

Reputation: 336

something like this should work

finalRDD.map(r => (r._1._1,r._1._2,r._2))

Upvotes: 2

Related Questions