CODEWITHSUNDEEP
scalaparallel-processingsbt
Karlo
Karlo

Reputation: 1708

How to control the number of parallel processes with .par in Scala?

I am using the par.map expression to execute processes in parallel in Scala (SBT).

Consider list.par.map(function(_)) (I am preparing an MWE). This means that function(_) should be applied to all the elements of the list in a parallel fashion. In my example, list has 3 elements. But Scala executes only function(list(1)) and function(list(2)) in parallel, and only afterwards function(list(3)).

Is there a reason for this behaviour? Is there a relation with the fact that the programme is executed on a two-core processor? Or how could you impose to execute all three things in parallel?

Upvotes: 1

Views: 2854

Answers (1)

Danny
Danny

Reputation: 541

This question has been asked before:

and is well documented:

what you want is something like:

var parallelList = list.par
parallelList.tasksupport = new ForkJoinTaskSupport(
     new scala.concurrent.forkjoin.ForkJoinPool(parlevel))
parallelList.map(function(_))

That said if your running on a 2 core processor you only have two threads (unless the cores are hyper threaded of course) meaning you can't have more than 2 parallel operations at once.

Upvotes: 5

Related Questions