CODEWITHSUNDEEP
scalafuture
MaatDeamon
MaatDeamon

Reputation: 9761

Parallel operation on list with future

I was just wondering as a general advice if the following made sense.

I have a list, that I need to filter according to the following criteria: let's say the list contain things of type A, B, C and D and I want to take n0 elt of A, n1 elt of B, n2 elt of C and n3 elt of D and then make one list out of it.

The iterative approach is pretty clean (i.e. going over the all list, using 4 counters, adding elt to each list until each respective counter reached it limit i.e. n1, n2, n3, n4), but a colleague at work told me to take advantage of multiple cpu and parallelize the operation using future.

In other words, launching 4 future operation that filter the list, and drop if it applies (i.e.resultinglist > nx), "resultinglist.size - n0 or n1 or n2 or n3 or n4". then await the result and combine the list.

I think this is an overkill for something we use to do iteratively pretty easily. I just wonder what people think about that. Yes I can run a test and compare the speed, but it raise the question of, when exactly can we ensure that we are taking advantage of the multiple cpu architecture. Because indeed I understand the motivation behind the suggestion. However, I did not know how to tell that it might be counter productive. We were both stuck in debate and enable to state if it is good situation or a bad situation to use parallelization. In other words, we did not have a criterion. Is testing the only way to know ?

Upvotes: 2

Views: 125

Answers (2)

Biswanath
Biswanath

Reputation: 9185

If you are bothered about the four futures you can use Parallel collection which is pretty easy to work with and you don't need make lot of changes from you non-parallel version.

Again to whether to go parallel or not will depend on other factors like, how large is the list,whether the operation which you will be doing on each element does have some contention.

Also you might find this paper on parallel collection by Martin Odersky and others interesting .

Upvotes: 1

nattyddubbs
nattyddubbs

Reputation: 2095

Avoid premature optimization.

If you benchmark and find that what you are doing is too slow for your requirement then incorporate the parallel version and do some benchmarking.

Upvotes: 1

Related Questions