CODEWITHSUNDEEP
pythonpython-3.xterminalparallel-processinggil
Avinash Shenoy
Avinash Shenoy

Reputation: 63

Does GIL affect parallel processing of a python script in separate terminal windows?

I am trying to understand Python's GIL. I recently had an assignment where I had to compare the execution times of a certain task performed using different algorithms of different time complexities on multiple input files. I ran a python script to do the same, but I used separate terminal windows on macOS to run the same python script for different input files.

I also ran it all in one terminal window, one after the other, for each input file. The CPU time for this was lower for each execution as compared to the previous approach with multiple windows where each program took twice as long but ran all at once. (Note : there were 4 terminal windows in the previous approach and the python script only ran an a.out executable compiled with clang on macOS and stored the output in different files).

Can anyone explain why running them in parallel lead to each program being slower? Did they run on separate cores or did the GIL lead to each program being slower than it would if I run it one at a time in one terminal window?

Upvotes: 1

Views: 266

Answers (1)

cco
cco

Reputation: 6281

Each terminal window will start a new python interpreter, each of which has its own GIL. The difference is probably due to contention for some resource at the OS level (disk i/o, memory, cpu cycles).

Upvotes: 2

Related Questions