CODEWITHSUNDEEP
scala
iwein
iwein

Reputation: 26171

How does mapValues translate to parallel collections in Scala

I'm trying to do a simple aggregation in a collection I've grouped. Like this:

case class Foo(key:String, value:Int)
val minPerKey = lotsOfFoos.groupBy(_.key).mapValues(_.minBy(_.value))

Now I want to do the same in parallel but ParMap doesn't have a mapValues function.

Why is this so, and how should I do it alternatively?

Upvotes: 1

Views: 222

Answers (2)

derekjw
derekjw

Reputation: 386

Scala 2.10 has mapValues for ParMap.

But there isn't a whole lot of benefit to using mapValues with a ParMap. The mapValues method just gives you a new Map that applies your function when reading values, it doesn't actually change the values in the Map. So you would need to do some other action that would force the creation of a new Map with the mapped values.

Upvotes: 3

iwein
iwein

Reputation: 26171

Since mapValues is not present on ParMapLike, there are two options left:

  1. Just use map: .map{case (key, value) => (key->value.minBy(_.value)) }
  2. Get out of the parallel universe: .toMap.mapValues(_.minBy(_.value))

I don't know why mapValues isn't just implemented on ParMapLike

Upvotes: 1

Related Questions