CODEWITHSUNDEEP
scalacollectionsparallel-processingparallel-collections
Omid
Omid

Reputation: 1989

Scala different parallel structures

I tested parallel collections on Scala vs simple collection, here is my code:

def parallelParse()
  {
    val adjs = wn.allSynsets(POS.ADJECTIVE).par
    adjs.foreach(adj => {
      parse(proc.mkDocument(adj.getGloss))
    })
  }

  def serialParse()
  {
    val adjs = wn.allSynsets(POS.ADJECTIVE)
    adjs.foreach(adj => {
      parse(proc.mkDocument(adj.getGloss))
    })
  }

The parallel collection speed up about 3 times. What other option do I have in Scala to make it even faster in parallel, I would be happy to test them and put the results here.

Upvotes: 1

Views: 71

Answers (1)

axel22
axel22

Reputation: 32335

You can use futures to start asynchronous computations. You could do:

import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
val futures = wn.allSynsets(POS.ADJECTIVE).map(adj => Future {
  parse(proc.mkDocument(adj.getGloss))
})
futures.foreach(f => Await.ready(f, Duration.Inf))

Depending on the amount of work on each element in allSynsets and the number of elements in allSynsets (too many elements -> too many futures -> more overheads), you could get worse results with futures.

To ensure that you are benchmarking correctly, consider using the inline benchmarking feature of ScalaMeter 0.5:

http://scalameter.github.io/home/gettingstarted/0.5/inline/index.html

You could also use actors to achieve this, but it would require a bit more plumbing.

Upvotes: 1

Related Questions