adrian
adrian

Reputation: 2376

Spark: Expansion of RDD(Key, List) to RDD(Key, Value)

So I have an RDD of something like this

RDD[(Int, List)]]

Where a single element in the RDD looks like

(1, List(1, 2, 3))

My question is how can I expand the key value pair to something like this

(1,1)
(1,2)
(1,3)

Thank you

Upvotes: 6

Views: 2504

Answers (2)

etov
etov

Reputation: 3024

And in Python (based on @seanowen's answer):

rdd.flatMap(lambda x: map(lambda e: (x[0], e), x[1]))

Upvotes: 5

Sean Owen
Sean Owen

Reputation: 66891

rdd.flatMap { case (key, values) => values.map((key, _)) }

Upvotes: 7

Related Questions