Comparison of multi-threading models in Julia >=1.3 and Python 3.x

Question

I would like to understand, from the user point of view, the differences in multithreading programming models between Julia >= 1.3 and Python 3.

Is there one that is more efficient than the other (in the sense that rising the thread numbers reduces more the computational time) ? In which situations (e.g. one model may have an edge, but only on computational or memory intensive tasks) ?

Is one that is more practical/provide higher level functions than the other ?

Is one that is more flexible than the other (e.g. it can be applied to a wider set of cases) ?

Answer 1

There are several differences between the languages with Julia providing many levels of functionality on this what you can find in Python. You have the following types of parallelism (I am discussing here the standard language features not functionality available via external libraries):

1. SIMD (signle-instruction-multiple-data) feature of CPUs

• Julia: combine @simd with @inbounds (see https://docs.julialang.org/en/v1/manual/performance-tips/)
• Python: not supported

2. Green threads (also called Coroutines). (This is not an actual threading - but allows to use one system thread across many tasks. This is particularly useful to parallelize IO operations such as web scraping or inter-process communication - for an example if one task is waiting for IO, another tasks can execute in parallel.)

• Julia: use a combination of @sync (to collect a group of tasks) and @async (to spawn new tasks) macros (for more details see https://docs.julialang.org/en/v1/manual/parallel-computing/)
• Python: use asyncio (for more details see https://docs.python.org/3/library/asyncio-task.html)

3. Multihreading: run several tasks in parallel within a single process (and shared memory) across several system threads:

• Julia: use Threads.@threads macro to parallelize loops and Threads.@spawn to launch tasks on separate system threads. Use locks or atomic values to control the parallel execution. (for more details see https://docs.julialang.org/en/v1/manual/parallel-computing/)

• Python: not useful for CPU-dominated tasks due to GIL (global-interpreter-lock) (see the comment by @Jim below)

4. Multi-processing

• Julia: use macros from the Distibuted package to parallelize loops and spawn remote processes (for more details see https://docs.julialang.org/en/v1/manual/parallel-computing/)

• Python: use multiprocessing library - for more details see https://docs.python.org/3.8/library/multiprocessing.html