Concurrency , Multi-processing, Parallelism, Multi-threading confusion between how they can relate

Question

Please correct me if I am wrong on my explanations and my question, I am trying to figure out how to word it all correctly.

1) Can multi-threading = parallelism?

I read that parallelism is running things at same time, not switching between them, but if you have a cpu that can take multiple processes and threads from processes, and run them on multiple cpus/cores, then can multi threading also be parallelism?

2) Follow up from above, if we assume the answer is no, does that mean that multi-threading is ALWAYS the same as Concurrency?

3) Can programs only be concurrent if they support multi-threading?, let's take PHP, I read it does not support multi-threading, so can a PHP execution still support Concurrency within itself if it can't spin up other threads? I hope this question made sense... basically, how can a program that can't support threads be running concurrency within its own execution?

4) Is multi-processing always considered parallelism?

Sorry for the mess of questions, not sure how to word them correctly.

Answer 1

Concurrency can be defined as the composition and interleaving of independently executing tasks.

Parallelism is the simultaneous execution of independent tasks.

A concurrent model can be parallel if the independent tasks are run at the same time. In other words, parallelism is a subset of concurrency.

When you are cooking a meal in your kitchen you are applying a concurrent model. Even though you can focus on a single task at a time, you keep interleaving among several (turn on the oven, cut some vegetable, place the meal in the oven once hot, take care of the sauce). All these operations are independent from each other but their execution sequence is important (you cannot put the meal in the oven before it gets hot). An optimal concurrent strategy can make the difference in the amount of time needed for cooking your meal.

Adding a second person to help you out is a parallel working model. It might make things faster or might not. It depends on many factors such as the recipe, the amount of tools available and the size of the kitchen. It certainly makes things harder as now you need also to co-ordinate two people while before you were just making sure the sequence of operations to be correct.

Hoping the definition of concurrency/parallelism to be clear, we need to be honest with ourselves and realize that the above models are guidelines and their implementation depends on several factors such as the hardware, the Operating System and the language platform.

Threads are often implemented by modern Operating Systems as unit of execution consisting of a dedicated stack and set of CPU registries. This allows the OS to switch between them rapidly without the need of releasing the entire process memory space. Most modern OSes can run threads in parallels if multiple CPU cores are available but this has not been always the case. Until 2011 for example, Linux was not allowing the simultaneous execution of threads belonging to the same process via its BKL.

Moreover even if the OS supports the parallel execution of threads belonging to the same process, not all language platforms allow that. The main Python interpreter for example, still denies the parallel execution of threads via its GIL.

As a general guidelines, threads can be considered concurrent in most of the cases. They might be parallel or not according to the platform.

Same issue applies to processes. The first versions of smartphone OSes were providing a very limited concurrency support and almost no parallelism. IoT device OSes might still be affected by similar issues.

Generally on hardware providing multiple CPU cores, multiprocessing allows parallelism in most of the cases.

There are other ways to achieve concurrency and even parallelism without the need of relying on OS specific facilities.

Erlang/Elixr for example can run multiple processes in parallel. Erlang processes are neither OS processes nor threads. They abstract concurrency in an implementation of their own.

Go introduced the goroutine as lightweight units of execution.

Python asyncio module allows the developer to interleave operations effectively enabling concurrent execution of code.