Reputation: 3056
I'm running some computationally heavy simulation in (home-made) C-based python extensions. Occasionally I get stuff wrong and would like to terminate a simulation. However, Ctrl-C doesn't seem to have any effect (other than printing ^C
to the screen) so I have to kill the process using kill
or the system monitor.
As far as I can see python just waits for the C extension to finish and doesn't really communicate with it during this time.
Is there a way to make this work?
Update: The main answers (for my specific problem) turned out to be:
1. rewrite the code to regularly pass control back to the caller (answer Allowing Ctrl-C to interrupt a python C-extension below), or
2. Use PyErr_CheckSignals()
(answer https://stackoverflow.com/a/33652496/423420 below)
Upvotes: 25
Views: 7752
Reputation: 96
There is an alternative way to solve this problem if you do not want your C Extension (or ctypes DLL) to be tied to Python, such as a case where you want to create a C library with bindings in multiple languages, you must allow your C Extension to run for long periods, and you can modify the C Extension:
Include the signal header in the C Extension.
#include <signal.h>
Create a signal handler typedef in the C Extension.
typedef void (*sighandler_t)(int);
Add signal handlers in the C extension that will perform the actions necessary to interrupt any long running code (set a stop flag, etc.), and save the existing Python signal handlers.
sighandler_t old_sig_int_handler = signal(SIGINT, your_sig_handler);
sighandler_t old_sig_term_handler = signal(SIGTERM, your_sig_handler);
Restore the existing signal handlers whenever the C extension returns. This step ensures that the Python signal handlers are re-applied.
signal(SIGINT, old_sig_int_handler);
signal(SIGTERM, old_sig_term_handler);
If the long-running code is interrupted (flag, etc.), return control to Python with a return code indicating the signal number.
return SIGINT;
In Python, send the signal received in the C extension.
import os
import signal
status = c_extension.run()
if status in [signal.SIGINT, signal.SIGTERM]:
os.kill(os.getpid(), status)
Python will perform the action you are expecting, such as raising a KeyboardInterrupt for SIGINT.
Upvotes: 4
Reputation: 6322
Not elegant, but the only approach I found that also interrupts external library calls in C++ and kills any running child processes.
#include <csignal>
#include <pybind11/pybind11.h>
void catch_signals() {
auto handler = [](int code) { throw std::runtime_error("SIGNAL " + std::to_string(code)); };
signal(SIGINT, handler);
signal(SIGTERM, handler);
signal(SIGKILL, handler);
}
PYBIND11_MODULE(example, m)
{
m.def("some_func", []()
{
catch_signals();
// ...
});
}
import sys
from example import some_func
try:
some_func()
except RuntimeError as e:
if "SIGNAL" in str(e):
code = int(str(e).rsplit(" ", 1)[1])
sys.exit(128 + code)
raise
Upvotes: 3
Reputation: 414585
However, Ctrl-C doesn't seem to have any effect
Ctrl-C
in the shell sends SIGINT
to the foreground process group. python
on receiving the signal sets a flag in C code. If your C extension runs in the main thread then no Python signal handler will be run (and therefore you won't see KeyboardInterrupt
exception on Ctrl-C
) unless you call PyErr_CheckSignals()
that checks the flag (it means: it shouldn't slow you down) and runs Python signal handlers if necessary or if your simulation allows Python code to execute (e.g., if the simulation uses Python callbacks). Here's a code example of an extension module for CPython created using pybind11 suggested by @Matt:
PYBIND11_MODULE(example, m)
{
m.def("long running_func", []()
{
for (;;) {
if (PyErr_CheckSignals() != 0)
throw py::error_already_set();
// Long running iteration
}
});
}
If the extension runs in a background thread then it is enough to release GIL (to allow Python code to run in the main thread that enables the signal handlers to run). PyErr_CheckSignals()
always returns 0
in a background thread.
Related: Cython, Python and KeybordInterrupt ingored
Upvotes: 22
Reputation: 179552
Python has a signal handler installed on SIGINT
which simply sets a flag that is checked by the main interpreter loop. For this handler to work properly, the Python interpreter has to be running Python code.
You have a couple of options available to you:
Py_BEGIN_ALLOW_THREADS
/Py_END_ALLOW_THREADS
to release the GIL around your C extension code. You cannot use any Python functions when not holding the GIL, but Python code (and other C code) may run concurrently with your C thread (true multithreading). A separate Python thread can execute alongside the C extension and catch Ctrl+C signals.SIGINT
handler and call the original (Python) signal handler. Your SIGINT
handler can then do whatever it needs to do to cancel the C extension code and return control to the Python interpreter.Upvotes: 8
Reputation: 1
I would redesign the C extensions so that they don't run for a long period.
So, split them into more elementary steps (each running for a short period of time, e.g. 10 to 50 milliseconds), and have these more elementary steps called by Python code.
continuation passing style might be relevant to understand, as a programming style...
Upvotes: 0