CODEWITHSUNDEEP
pythonmultithreadingconcurrencyhardware
Neil
Neil

Reputation: 3301

How do Python processes and threads map onto hardware threads?

My question is similar to this one, but I'm looking for up-to-date information on how this works with:

Assuming a CPU with 4 cores and 8 threads, for example, and given this code:

from multiprocessing.dummy import Pool as ThreadPool
from multiprocessing import Pool as ProcessPool


with ThreadPool(number_of_threads) as pool:
    pool.map(some_function, some_iterable)

with ProcessPool(number_of_processes) as pool:
    pool.map(some_function, some_iterable)

I'm wondering how these threads and processes will map onto the CPU cores and threads, if at all. For example :

I always use ThreadPool for IO bound and ProcessPool for CPU bound. With ProcessPool, I've found that using the same number_of_processes as CPU cores works well. But I don't know with ThreadPool what my benchmark should be.

Upvotes: 2

Views: 706

Answers (1)

dm03514
dm03514

Reputation: 55972

I think a big disclaimer is: Theoretical "how it should work" will only get so far. For actual performance comparisons, ie the performance difference between a thread pool of 4, 6, 8, or any number of threads, the only concrete answers come from benchmarking on your target system using realistic workloads.

One of the most important questions to inform the answers here is:

Does it make a difference to any of the above whether some_function is IO bound or CPU bound?

This is the most important perf question IMO. What is your workload (some_iterable)? Is it I/O Bound or CPU bound? I/O bound is making http requests, or querying a database, or filesystem. CPU bound is doing some sort of computation, hashing, adding, parsing, etc. Most workloads (in my experiences are a mix, but if I had to say based on experience, favor I/O). If the workload is I/O and some_iterable is doing synchronous I/O than you could very probably scale performance by creating larger worker pools, Since most of the time a process/thread is executing will be spent waiting on I/O!

If number_of_processes is 8, will each process likely end up on a cpu thread?

Kind of :p. You have 8 physical threads. There are 8 processes/main threads of each pool, plus the thread of your executing program, plus all the other processes/threads of your OS :). Your processors are saturated. If you have a CPU bound workload they will be saturated and you will likely not see an increase in performance over a pool size of 7-8. If your workload is I/O bound, you will probably still see performance increases with a pool size greater than the # of processors you have.

If number_of_threads is 8, will each thread likely end up on a cpu thread?

Same as processor question above. Yes they will most likely end up on a CPU thread. If your workload is I/O bound increasing a pool size will probably still yield a performance increase for some time.
For CPU bound workfloads this is where things get complicated. Python's GIL prevents python from executing multiple python bytecode at once. And even though you will have enough physical CPU threads to execute your program when you need to only a single thread can execute at a time! I would be suprised if a CPU bound workfload, a number_of_threads = 8 gives better performance than a number_of_threads = 4 for CPU bound workloads!

What are the implications if number_of_threads are much higher than the number of cores/cpu threads?

For I/O bound workfloads, nothing! you'll hit a performance limit at sometime, based on either your machines ability to schedule and execute a huge number of threads or the upstream services ability to service all the requests you're making. For CPU bound workloads see the answer above (you'll hit limits way faster because of GIL).

References:

Also in my experiences doing primarily http based services, thinking about logical cores (hardware threads) as being the same as physical cores, hasn't ever come back to bite me. So in your case I would just consider that you have 8 cores available. The distinction is probably not important for your workload (shot in the dark)?

Upvotes: 1

Related Questions