CODEWITHSUNDEEP
pythonmultithreading
e wagness
e wagness

Reputation: 301

How does python handle thread locking / context switching?

I watched an excellent presentation on the GIL, and how when running in the interpreter only 1 single thread can run at a time. It also seemed that python is not very intelligent about switching between threads.

If i am threading some operation that only runs in the interpreter, and it is not particularly CPU heavy, and I use a thread lock where only 1 thread can run at a time for this relatively short interpreter-bound operation, will that lock actually make anything run slower? as opposed to if the lock were not necessary and all threads could run concurrently.

If all but 1 threads are locked, will the python interpreter know not to context switch?

Edit: by 'making things run slower' I mean if python is context switching to a bunch of locked threads, that will (maybe) be a performance decrease even if the threads don't actually run

Upvotes: 7

Views: 9953

Answers (2)

DisappointedByUnaccountableMod
DisappointedByUnaccountableMod

Reputation: 6826

python is not very intelligent about switching between threads

Python threads work a certain way :-)

if I use a thread lock where only 1 thread can run at a time... will that lock actually make anything run slower

Err, no because there is nothing else runnable, so nothing else could run slower.

If all but 1 threads are locked, will the python interpreter know not to context switch?

Yes. The kernel knows which threads are runnable. If no other threads can run then logically speaking (as far as the thread is concerned) the python interpreter won't context switch away from the only runnable thread. The thread doesn't know when it has been switched away from (how can it, it isn't running).

Upvotes: 5

tipsqueal
tipsqueal

Reputation: 203

Larry Hastings (a core CPython Developer) has a great talk that covers this subject called "Python's Infamous GIL". If you skip to 11:40ish he gives the answer to your question.

From the talk: The way Python threads work with the GIL is with a simple counter. With every 100 byte codes executed the GIL is supposed to be released by the thread currently executing in order to give other threads a chance to execute code. This behavior is essentially broken in Python 2.7 because of the thread release/acquire mechanism. It has been fixed in Python 3.

When you use a thread lock Python will only execute the threads that are not locked. So if you have several threads sharing 1 lock, then only one thread will execute at the same time. Python will not start executing a locked thread until the thread can acquire the lock. Locks are there so you can have shared state between threads without introducing bugs.

If you have several threads and only 1 can run at a time because of a lock, then in theory your program will take longer to execute. In practice you should benchmark, because the results will surprise you.

Upvotes: 14

Related Questions