CODEWITHSUNDEEP
javamultithreadingparallel-processing
Abhash Upadhyaya
Abhash Upadhyaya

Reputation: 727

Limitations of parallelizing heterogeneous tasks

I was reading Java Concurrency in Practice and a paragraph came across which said something like this:-

Under 6.3.4 Limitations of parallelizing heterogeneous tasks

A further problem with dividing heterogeneous tasks among multiple workers is that the tasks may have disparate sizes. If you divide tasks A and B between two workers but A takes ten times as long as B, you’ve only speeded up the total process by 9%. Finally, dividing a task among multiple workers always involves some amount of coordination overhead; for the division to be worthwhile, this overhead must be more than compensated by productivity improvements due to parallelism.

Now if you go through the bold part, is it correct to say that worker threads' co-ordination overhead must be more than the performance improvement I achieve using parallelism of threads?

Could some one please help me understand it?

Upvotes: 2

Views: 99

Answers (1)

JonK
JonK

Reputation: 2108

No, the bolded bit changes the meaning of the sentence quite significantly:

Finally, dividing a task among multiple workers always involves some amount of coordination overhead; for the division to be worthwhile, this overhead must be more than compensated by productivity improvements due to parallelism.

If that was simply "must be more than the productivity improvements..." then you would be right. What the above is saying is that the performance penalty you incur by dividing the tasks among your worker threads (the overhead), must be less than the performance increase you gain by doing it at all.

For instance, if your overhead is 10 seconds and your performance gain is only 5 seconds, then parallelising the operation has actually decreased your overall performance. If your overhead is 10 seconds but the performance gain is 50 seconds, you've ended up with a 40 second reduction in the overall execution time (i.e. an improvement).

Another way of saying it would be "...this overhead must be offset by the productivity improvements due to parallelism.".

Upvotes: 3

Related Questions