CODEWITHSUNDEEP
pythonmultithreadingamazon-web-servicesmultiprocessing
Peter Franek
Peter Franek

Reputation: 687

Python Multiprocessing: #cores versus #CPU's

It seems to me that using the python multiprocessing Pool.map as described here parallelizes the process to some extent between different cores of one CPU, but I have the feeling that there is no speed-up reflecting more CPU's on a computer. If that's right, is there a way to effectively use the "Number of CPU's times number of cores in each CPU"?

(Admittedly, I may be wrong because my experiments are based on a virtual Amazon cloud machine with 16 virtual CPU's but I know it's not a "real computer".)

Upvotes: 0

Views: 209

Answers (1)

Serge Ballesta
Serge Ballesta

Reputation: 148870

More exactly, by default the number of processes will be the number of cores presented by the OS. If the computer uses more than one CPU, the OS should present the total number of cores to Python. But anyway, you can always force the number of process to a smaller value is you do not want to use all the resources from the machine (if it is running a background server for example) or to a higher value if the task is not CPU bound but IO bound for example.

Upvotes: 1

Related Questions