CODEWITHSUNDEEP
apache-pig
Gaurav Phapale
Gaurav Phapale

Reputation: 979

Pig: flatten a tuple like bag

I have a tuple containing unknown number of fields. I want to flatten the tuple so that every element of the tuple creates a new row (the way Bag flattens)

e.g.
Input:

student = { (A, B, (C, D, E, F)) }

Output:

student = { (A, B, C),
            (A, B, D),
            (A, B, E),
            (A, B, F),
          }

How do I achieve this?

Upvotes: 1

Views: 720

Answers (1)

Gaurav Phapale
Gaurav Phapale

Reputation: 979

OK, got it working! Used LinkedIn's DataFu UDF library function TransposeTupleToBag to transpose tuples to bags, then flattened the bag.

Something like this: out = foreach student generate $0, $1, flatten(TransposeTupleToBag($2));

Upvotes: 3

Related Questions