CODEWITHSUNDEEP
pythonsignalspython-multithreadingpython-asyncio
highsciguy
highsciguy

Reputation: 2647

Python and signal handlers

I am in need for some clarification regarding signal handlers in python, as I do not precisely understand how they work, how I can use them and what the limitations are.

I intend to use USR signals on linux in order to communicate with a python program running in the background as a service.

I found that, as intended, the signals I send seem to be processed immediately in an asynchronous way.

Therefore, I used to think that the registered signal handlers run in their own threads which, I thought, would explain why the following code will print multiple lines of Signal handler called with signal 10 at once when sending signals in a loop

#!/usr/bin/python3.5
# This is the file signal_example.py

import signal, time, os

def handler(signum, frame):
    print('Signal handler called with signal', signum)
    time.sleep(20)

signal.signal(signal.SIGUSR1, handler)

time.sleep(100)


#!/usr/bin/bash
for ((i=0;i<100;i=i+1)); do 
    killall -s SIGUSR1 signal_example.py; 
done

However, documentation states (https://docs.python.org/3.4/library/signal.html) "Python signal handlers are always executed in the main Python thread, even if the signal was received in another thread.". Also, I do not see separate threads running on my system for the above example.

I am now wondering, what the actual implications and limitations of my approach to using signals are and how I can properly do relevant things such as communication between instances of the invoked signal handlers.

For example, in certain cases, I would like to be able to delay the execution of some code inside a signal handler until previous signals of the same kind have been processed. Also I do not understand how many signals will be processed "in parallel" and what happens when the "queue" is full ...

I have been looking a bit into Pyhton's asyncio, which seems to offer some control of asynchronous code, and also brings its own way to register signal handlers. It seems to offer some aids, however, it does not seem to help me a lot since the behavior that I see (signals beeing handled pretty much when they are received) is actually what I want. The use of asyncio signal handlers that I have seen appears to execute the signals in the event loop. In the partially blocking code that I am starting with, that could be too late. I could run some of it in separate threads (using the corresponding asyncio functions), but not make the entire code non-blocking.

Upvotes: 2

Views: 9215

Answers (1)

user4815162342
user4815162342

Reputation: 155580

I intend to use USR signals on linux in order to communicate with a python program running in the background as a service.

That sounds like a really bad idea. Here are several reasons why:

  • Unix signals are delivered asynchronously, which means you can get a signal while any library code is running, such as in the middle of a malloc call. For this reason, only a small number of functions are async-signal-safe, i.e. safe to invoke from a signal handler. Python code cannot execute inside a signal handler at all, so the handler set by signal.signal doesn't execute the Python function, but simply sets a global flag. This flag is checked by the main thread from time to time, which executes the Python handler function. This means that there is no guarantee that the signal will be delivered immediately, i.e. you cannot rely on the signal delivery guarantees provided by the operating system.

  • A signal may arrive during the execution of a handler of a previous signal. This is why you are seeing multiple lines being printed, it is the artifact of the signal handler itself being interrupted by a signal and reentering.

  • Unless you set up signal handlers using specialized POSIX calls (and handlers set up using those calls can't directly execute Python code), signals will not be queued. Even when they are queued, except for non-real-time signals, the order of signals between generation and delivery is not preserved.

  • Signals interact really badly with multi-threaded code, even in pure C.

If you have a service that you need to communicate with, you could have a thread that reads from a named pipe or listens on a socket. Writing to the pipe, or opening the socket, would serve as a signal to the process. This also allows passing additional information to the service.

Upvotes: 4

Related Questions