Exception handled surprisingly in PyQt/Pyside slots

Question

Problem: When exceptions are raised in slots, invoked by signals, they do not seem to propagate as usual through Pythons call stack. In the example code below invoking:

• on_raise_without_signal(): Will handle the exception as expected.
• on_raise_with_signal(): Will print the exception and then unexpectedly print the success message from the else block.

Question: What is the reason behind the exception being handled surprisingly when raised in a slot? Is it some implementation detail/limitation of the PySide Qt wrapping of signals/slots? Is there something to read about in the docs?

PS: I initially came across that topic when I got surprising results upon using try/except/else/finally when implementing a QAbstractTableModels virtual methods insertRows() and removeRows().

---

# -*- coding: utf-8 -*-
"""Testing exception handling in PySide slots."""
from __future__ import unicode_literals, print_function, division

import logging
import sys

from PySide import QtCore
from PySide import QtGui


logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)


class ExceptionTestWidget(QtGui.QWidget):

    raise_exception = QtCore.Signal()

    def __init__(self, *args, **kwargs):
        super(ExceptionTestWidget, self).__init__(*args, **kwargs)

        self.raise_exception.connect(self.slot_raise_exception)

        layout = QtGui.QVBoxLayout()
        self.setLayout(layout)

        # button to invoke handler that handles raised exception as expected
        btn_raise_without_signal = QtGui.QPushButton("Raise without signal")
        btn_raise_without_signal.clicked.connect(self.on_raise_without_signal)
        layout.addWidget(btn_raise_without_signal)

        # button to invoke handler that handles raised exception via signal unexpectedly
        btn_raise_with_signal = QtGui.QPushButton("Raise with signal")
        btn_raise_with_signal.clicked.connect(self.on_raise_with_signal)
        layout.addWidget(btn_raise_with_signal)

    def slot_raise_exception(self):
        raise ValueError("ValueError on purpose")

    def on_raise_without_signal(self):
        """Call function that raises exception directly."""
        try:
            self.slot_raise_exception()
        except ValueError as exception_instance:
            logger.error("{}".format(exception_instance))
        else:
            logger.info("on_raise_without_signal() executed successfully")

    def on_raise_with_signal(self):
        """Call slot that raises exception via signal."""
        try:
            self.raise_exception.emit()
        except ValueError as exception_instance:
            logger.error("{}".format(exception_instance))
        else:
            logger.info("on_raise_with_signal() executed successfully")


if (__name__ == "__main__"):
    application = QtGui.QApplication(sys.argv)

    widget = ExceptionTestWidget()
    widget.show()

    sys.exit(application.exec_())

Answer 1

This way of handling expcetions is not a surprise taking into consideration that the Signal/Slot architecture proposes a loosely coupled intercation between the signals and the slots. This means that the signal should not be expecting anything to happen inside the slots.

Although timmwagener's solution was pretty clever, it should be used with caution. Probably the problem is not with how Exceptions are handled between Qt Connections, but that the signal/slot architecture is not ideal for your application. Also, that solution would not work if a slot from a different thread is connected, or a Qt.QueuedConnection is used.

A good way of tackling the problem of errors raised in slots is to determine that at the connection and not the emitting. Then the erros can be handled in a loosely coupled way.

class ExceptionTestWidget(QtGui.QWidget):

    error = QtCore.Signal(object)

    def abort_execution():
        pass

    def error_handler(self, err):
        self.error.emit(error)
        self.abort_execution()

(...)

def connect_with_async_error_handler(sig, slot, error_handler, *args,
                                     conn_type=None, **kwargs):                              

    @functools.wraps(slot)
    def slot_with_error_handler(*args):
        try:
            slot(*args)
        except Exception as err:
            error_handler(err)

    if conn_type is not None:
        sig.connect(slot_with_error_handler, conn_type)
    else:
        sig.connect(slot_with_error_handler)

This way, we would be complying to the requirements in the Qt5 docs, stating that you need to handle exceptions within the slot being invoked.

Throwing an exception from a slot invoked by Qt's signal-slot connection mechanism is considered undefined behaviour, unless it is handled within the slot

PS: This is just a sugestion based on a very small overview of your use case. There is no right/wrong way of solving this, I just wanted to bring out a different point of view : )

Answer 2

Thanks for answering guys. I found ekhumoros answer particularly useful to understand where the exceptions are handled and because of the idea to utilize sys.excepthook.

I mocked up a quick solution via context manager to temporarily extend the current sys.excepthook to record any exception in the realm of "C++ calling Python" (as it seems to happen when slots are invoked by signals or virtual methods) and possibly re-raise upon exiting the context to achieve expected control flow in try/except/else/finally blocks.

The context manager allows on_raise_with_signal to maintain the same control flow as on_raise_without_signal with the surrounding try/except/else/finally block.

---

# -*- coding: utf-8 -*-
"""Testing exception handling in PySide slots."""
from __future__ import unicode_literals, print_function, division

import logging
import sys
from functools import wraps

from PySide import QtCore
from PySide import QtGui


logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)


class ExceptionHook(object):

    def extend_exception_hook(self, exception_hook):
        """Decorate sys.excepthook to store a record on the context manager
        instance that might be used upon leaving the context.
        """

        @wraps(exception_hook)
        def wrapped_exception_hook(exc_type, exc_val, exc_tb):
            self.exc_val = exc_val
            return exception_hook(exc_type, exc_val, exc_tb)

        return wrapped_exception_hook

    def __enter__(self):
        """Temporary extend current exception hook."""
        self.current_exception_hook = sys.excepthook
        sys.excepthook = self.extend_exception_hook(sys.excepthook)

        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        """Reset current exception hook and re-raise in Python call stack after
        we have left the realm of `C++ calling Python`.
        """
        sys.excepthook = self.current_exception_hook

        try:
            exception_type = type(self.exc_val)
        except AttributeError:
            pass
        else:
            msg = "{}".format(self.exc_val)
            raise exception_type(msg)


class ExceptionTestWidget(QtGui.QWidget):

    raise_exception = QtCore.Signal()

    def __init__(self, *args, **kwargs):
        super(ExceptionTestWidget, self).__init__(*args, **kwargs)

        self.raise_exception.connect(self.slot_raise_exception)

        layout = QtGui.QVBoxLayout()
        self.setLayout(layout)

        # button to invoke handler that handles raised exception as expected
        btn_raise_without_signal = QtGui.QPushButton("Raise without signal")
        btn_raise_without_signal.clicked.connect(self.on_raise_without_signal)
        layout.addWidget(btn_raise_without_signal)

        # button to invoke handler that handles raised exception via signal unexpectedly
        btn_raise_with_signal = QtGui.QPushButton("Raise with signal")
        btn_raise_with_signal.clicked.connect(self.on_raise_with_signal)
        layout.addWidget(btn_raise_with_signal)

    def slot_raise_exception(self):
        raise ValueError("ValueError on purpose")

    def on_raise_without_signal(self):
        """Call function that raises exception directly."""
        try:
            self.slot_raise_exception()
        except ValueError as exception_instance:
            logger.error("{}".format(exception_instance))
        else:
            logger.info("on_raise_without_signal() executed successfully")

    def on_raise_with_signal(self):
        """Call slot that raises exception via signal."""
        try:
            with ExceptionHook() as exception_hook:
                self.raise_exception.emit()
        except ValueError as exception_instance:
            logger.error("{}".format(exception_instance))
        else:
            logger.info("on_raise_with_signal() executed successfully")


if (__name__ == "__main__"):
    application = QtGui.QApplication(sys.argv)

    widget = ExceptionTestWidget()
    widget.show()

    sys.exit(application.exec_())

Answer 3

As you've already noted in your question, the real issue here is the treatment of unhandled exceptions raised in python code executed from C++. So this is not only about signals: it also affects reimplemented virtual methods as well.

In PySide, PyQt4, and all PyQt5 versions up to 5.5, the default behaviour is to automatically catch the error on the C++ side and dump a traceback to stderr. Normally, a python script would also automatically terminate after this. But that is not what happens here. Instead, the PySide/PyQt script just carries on regardless, and many people quite rightly regard this as a bug (or at least a misfeature). In PyQt-5.5, this behaviour has now been changed so that qFatal() is also called on the C++ side, and the program will abort like a normal python script would. (I don't know what the current situation is with PySide2, though).

So - what should be done about all this? The best solution for all versions of PySide and PyQt is to install an exception hook - because it will always take precedence over the default behaviour (whatever that may be). Any unhandled exception raised by a signal, virtual method or other python code will firstly invoke sys.excepthook, allowing you to fully customise the behaviour in whatever way you like.

In your example script, this could simply mean adding something like this:

def excepthook(cls, exception, traceback):
    print('calling excepthook...')
    logger.error("{}".format(exception))

sys.excepthook = excepthook

and now the exception raised by on_raise_with_signal can be handled in the same way as all other unhandled exceptions.

Of course, this does imply that best practice for most PySide/PyQt applications is to use largely centralised exception handling. This often includes showing some kind of crash-dialog where the user can report unexpected errors.

Answer 4

According to the Qt5 docs you need to handle exceptions within the slot being invoked.

Throwing an exception from a slot invoked by Qt's signal-slot connection mechanism is considered undefined behaviour, unless it is handled within the slot

State state;
StateListener stateListener;

// OK; the exception is handled before it leaves the slot.
QObject::connect(&state, SIGNAL(stateChanged()), &stateListener, SLOT(throwHandledException()));
// Undefined behaviour; upon invocation of the slot, the exception will be propagated to the
// point of emission, unwinding the stack of the Qt code (which is not guaranteed to be exception safe).
QObject::connect(&state, SIGNAL(stateChanged()), &stateListener, SLOT(throwUnhandledException()));

If the slot was invoked directly, like a regular function call, exceptions may be used. This is because the connection mechanism is bypassed when invoking slots directly

In the first case you call slot_raise_exception() directly, so this is fine.

In the second case you are invoking it via the raise_exception signal, so the exception will only propagate up to the point where slot_raise_exception() is called. You need to place the try/except/else inside slot_raise_exception() for the exception to be handled correctly.