CODEWITHSUNDEEP
scalahadoopmapreducecascadingscalding
Savage Reader
Savage Reader

Reputation: 417

groupBy toList element order

I have a RichPipe with several fields, let's say:

'sex
'weight
'age

I need to group by 'sex and then get a list of tuples ('weight and 'age). I then want to do a scanLeft operation on the list for each group and get a pipe with 'sex and 'result. I currently do this by doing

pipe.groupBy('sex) {_.toList('weight -> 'weights).toList('age - 'ages)}

and then zipping the two lists together. I'm not sure this is the best possible way, and also I'm not sure if the order of the values in the lists is the same, so that when I zip the two lists the tuples don't get mixed up with wrong values. I found nothing about this in the documentation.

Upvotes: 1

Views: 421

Answers (1)

Savage Reader
Savage Reader

Reputation: 417

Ok, so it looks like I've answered my own question.

You can simply do

pipe.groupBy('sex) {_.toList[(Int, Int)](('weight, 'age) -> 'list)}

which results in a list of tuples. Would've saved me a lot of time if the Fields API Reference mentioned this.

Upvotes: 1

Related Questions