Simple, non-blocking background calls

Question

I'm writing an PySide application that communicates with hardware over a serial connection.

I have a button to start the device command and a label to show the result to the user. Now some devices take a long time (multiple seconds) to answer a request, which freezes the GUI. I am searching for a simple mechanism to run the call in a background thread or similar.

I created a short example of what I am trying to accomplish:

import sys
import time
from PySide import QtCore, QtGui


class Device(QtCore.QObject):

    def request(self, cmd):
        time.sleep(3)
        return 'Result for {}'.format(cmd)


class Dialog(QtGui.QDialog):

    def __init__(self, device, parent=None):
        super().__init__(parent)
        self.device = device

        self.layout = QtGui.QHBoxLayout()
        self.label = QtGui.QLabel('--')
        self.button = QtGui.QPushButton('Go')
        self.layout.addWidget(self.label)
        self.layout.addWidget(self.button)
        self.setLayout(self.layout)

        self.button.clicked.connect(self.go)

    def go(self):
        self.button.setEnabled(False)

        # the next line should be called in the
        # background and not freeze the gui
        result = self.device.request('command')

        self.label.setText(result)
        self.button.setEnabled(True)


if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    dev = Device()
    win = Dialog(device=dev)
    win.show()
    win.raise_()
    app.exec_()

What I wish to have is some kind of function like:

result = nonblocking(self.device.request, 'command')

Exceptions should be raised as if I called the function directly.

Any ideas or recommendations?

Answer 1

What I wish to have is some kind of function like:

    result = nonblocking(self.device.request, 'command')

What you are asking for here is effectively impossible. You cannot have a non-blocking call return a result immediately! That, by definition, would be a blocking call.

A non-blocking call would look something like:

self.thread = inthread(self.device.request, 'command', callback)

Where self.device.request runs the callback function/method at the completion of the serial request. However, naively calling callback() from within the thread will make the callback run in the thread, which is very very bad if you were to make calls to the Qt GUI from the callback method (Qt GUI methods are not thread safe). As such, you need a way of running the callback in the MainThread.

I've had a similar need for such functions, and have created (with a colleague) a library (called qtutils) which has some nice wrapper functions in it (including an implementation of inthread). It works by posting an event to the MainThread using QApplication.postEvent(). The event contains reference to a function to be run, and the event handler (which resides in the main thread) executes that function. So your request method would then look like:

def request(self, cmd, callback):
    time.sleep(3)
    inmain(callback, 'Result for {}'.format(cmd))

where inmain() makes posts an event to the main thread and runs callback(data).

Documentation for this library can be found here, or alternatively, you can role your own based on the outline above. If you choose to use my library, it can be installed via pip or easy_install.

Note: One caveat with this method is that QApplication.postEvent() leaks memory in PySide. That's why I now use PyQt!

Answer 2

Threading is the best way to do that. Python threads are really easy to use too. Qt threads don't work the same way as python threads. http://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/

I just use python threads. Also if you are using a serial port connection you may want to split the data off into a queue which is thread safe.

import time
import threading
import queue
import serial

def read_serial(serial, storage):
    while True:
        value = serial.readline()
        storage.put(value) # or just process the data

ser = serial.SerailPort("Com1", 9600)
stor = queue.Queue()
th = threading.Thread(target=read_serial, args=(ser, stor))
th.start()
for _ in range(10):
    time.sleep(1)
th.join(0)
ser.close()
# Read queue

The other thing you can do is use multiple inheritance for the serial port and the QObject. This allows you to use Qt signals. Below is a very rough example.

class SerialThread(object):
    def __init__(self, port=None, baud=9600):
        super().__init__()

        self.state = threading.Condition() # Notify threading changes safely
        self.alive = threading.Event() # helps with locking

        self.data = queue.Queue()
        self.serial = serial.Serial()
        self.thread = threading.Thread(target=self._run)

        if port is not None:
            self.connect(port, baud)
    # end Constructor

    def connect(self, port, baud):
        self.serial.setPort(port)
        self.serial.setBaudrate(baud)
        self.serial.open()

        with self.state:
            self.alive.set()
        self.thread.start()
    # end connect

    def _run(self):
        while True:
            with self.state:
                if not self.alive.is_set():
                    return

            self.read()
    # end _run

    def read(self):
        serstring = bytes("", "ascii")
        try:
            serstring = self.serial.readline()
        except:
            pass
        else:
            self.process_read(serstring)
        return serstring # if called directly
    # end read

    def process_read(self, serstring):
        if self.queue.full():
            self.queue.get(0) # remove the first item to free up space
        self.queue.put(serstring)
    # end process_read

    def disconnect(self):
        with self.state:
            self.alive.clear()
            self.state.notify()
        self.thread.join(0) # Close the thread
        self.serial.close() # Close the serial port
    # end disconnect
# end class SerialThread

class SerialPort(SerialThread, QtCore.QObject):
    data_updated = QtCore.Signal()

    def process_read(self, serstring):
        super().process_read(serstring)
        self.data_updated.emit()
    # end process_read
# end class SerialPort


if __name__ == "__main__":
    ser = SerialPort()
    ser.connect("COM1", 9600)
    # Do something / wait / handle data
    ser.disconnect()
    ser.queue.get() # Handle data

As always make sure you properly close and disconnect everything when you exit. Also note that a thread can only be run once, so you may want to look at a pausable thread example How to start and stop thread?. You can also just emit the data through a Qt Signal instead of using a queue to store the data.