CODEWITHSUNDEEP
pythonwindowsmultithreadingpython-multithreading
Neraste
Neraste

Reputation: 525

Stop multithreaded Python script on Windows

I have troubles with a simple multithreaded Python looping program. It should loop infinitely and stop with Ctrl+C. Here is an implementation using threading:

from threading import Thread, Event
from time import sleep

stop = Event()

def loop():
    while not stop.is_set():
        print("looping")
        sleep(2)

try:
    thread = Thread(target=loop)
    thread.start()
    thread.join()

except KeyboardInterrupt:
    print("stopping")
    stop.set()

This MWE is extracted from a more complex code (obviously, I do not need multithreading to create an infinite loop).

It works as expected on Linux, but not on Windows: the Ctrl+C event is not intercepted and the loop continues infinitely. According to the Python Dev mailing list, the different behaviors are due to the way Ctrl+C is handled by the two OSs.

So, it appears that one cannot simply rely on Ctrl+C with threading on Windows. My question is: what are the other ways to stop a multithreaded Python script on this OS with Ctrl+C?

Upvotes: 0

Views: 592

Answers (1)

abarnert
abarnert

Reputation: 366053

As explained by Nathaniel J. Smith in the link from your question, at least as of CPython 3.7, Ctrl-C cannot wake your main thread on Windows:

The end result is that on Windows, control-C almost never works to wake up a blocked Python process, with a few special exceptions where someone did the work to implement this. On Python 2 the only functions that have this implemented are time.sleep() and multiprocessing.Semaphore.acquire; on Python 3 there are a few more (you can grep the source for _PyOS_SigintEvent to find them), but Thread.join isn't one of them.

So, what can you do?

One option is to just not use Ctrl-C to kill your program, and instead use something that calls, e.g., TerminateProcess, such as the builtin taskkill tool, or a Python script using the os module. But you don't want that.

And obviously, waiting until they come up with a fix in Python 3.8 or 3.9 or never before you can Ctrl-C your program is not acceptable.

So, the only thing you can do is not block the main thread on Thread.join, or anything else non-interruptable.

The quick&dirty solution is to just poll join with a timeout:

while thread.is_alive():
    thread.join(0.2)

Now, your program is briefly interruptable while it's doing the while loop and calling is_alive, before going back to an uninterruptable sleep for another 200ms. Any Ctrl-C that comes in during that 200ms will just wait for you to process it, so that isn't a problem.

Except that 200ms is already long enough to be noticeable and maybe annoying.

And it may be too short as well as too long. Sure, it's not wasting much CPU to wake up every 200ms and execute a handful of Python bytecodes, but it's not nothing, and it's still getting a timeslice in the scheduler, and that may be enough to, e.g., keep a laptop from going into one of its long-term low-power modes.

The clean solution is to find another function to block on. As Nathaniel J. Smith says:

you can grep the source for _PyOS_SigintEvent to find them

But there may not be anything that fits very well. It's hard to imagine how you'd design your program to block on multiprocessing.Semaphore.acquire in a way that wouldn't be horribly confusing to the reader…

In that case, you might want to drag in the Win32 API directly, whether via PyWin32 or ctypes. Look at how functions like time.sleep and multiprocessing.Semaphore.acquire manage to be interruptible, block on whatever they're using, and have your thread signal whatever it is you're blocking on at exit.

If you're willing to use undocumented internals of CPython, it looks like, at least in 3.7, the hidden _winapi module has a wrapper function around WaitForMultipleObjects that appends the magic _PyOSSigintEvent for you when you're doing a wait-first rather than wait-all.

One of the things you can pass to WaitForMultipleObjects is a Win32 thread handle, which has the same effect as a join, although I'm not sure if there's an easy way to get the thread handle out of a Python thread.

Alternatively, you can manually create some kind of kernel sync object (I don't know the _winapi module very well, and I don't have a Windows system, so you'll probably have to read the source yourself, or at least help it in the interactive interpreter, to see what wrappers it offers), WaitForMultipleObjects on that, and have the thread signal it.

Upvotes: 1

Related Questions