How to architect a GUI application with UART comms which stays responsive to the user

Question

I'm writing an application in PyQt5 which will be used for calibration and test of a product. The important details:

• The product under test uses an old-school UART/serial communication link at 9600 baud.
• ...and the test / calibration operation involves communicating with another device which has a UART/serial communication link at 300 baud(!)
• In both cases, the communication protocol is ASCII text with messages terminated by a newline \r\n.

During the test/calibration cycle the GUI needs to communicate with the devices, take readings, and log those readings to various boxes in the screen. The trouble is, with the slow UART communications (and the long time-outs if there is a comms drop-out) how do I keep the GUI responsive?

The Minimally Acceptable solution (already working) is to create a GUI which communicates over the serial port, but the user interface becomes decidedly sluggish and herky-jerky while the GUI is waiting for calls to serial.read() to either complete or time out.

The Desired solution is a GUI which has a nice smooth responsive feel to it, even while it is transmitting and receiving serial data.

The Stretch Goal solution is a GUI which will log every single character of the serial communications to a text display used for debugging, while still providing some nice "message-level" abstraction for the actual logic of the application.

My present "minimally acceptable" implementation uses a state machine where I run a series of short functions, typically including the serial.write() and serial.read() commands, with pauses to allow the GUI to update. But the state machine makes the GUI logic somewhat tricky to follow; the code would be much easier to understand if the program flow for communicating to the device was written in a simple linear fashion.

I'm really hesitant to sprinkle a bunch of processEvents() calls throughout the code. And even those don't help when waiting for serial.read(). So the correct solution probably involves threading, signals, and slots, but I'm guessing that "threading" has the same two Golden Rules as "optimization": Rule 1: Don't do it. Rule 2 (experts only): Don't do it yet.

Are there any existing architectures or design patterns to use as a starting point for this type of application?

Answer 1

Qt is not only a library to make GUI but is a library that has other modules such as Qt Networt, Qt WebEngine, Qt Mqtt, etc. that work without blocking the Qt event loop, in its case it is best to use Qt SerialPort avoiding the problem that always brings threads. For example, in this answer I show the implementation of the same solution using Qt SerialPort and threads + pyserial.

Answer 2

Okay for the past few days I've been digging, and figured out how to do this. Since there haven't been any responses, and I do think this question could apply to others, I'll go ahead and post my solution. Briefly:

• Yes, the best way to solve this is with with PyQt Threads, and using Signals and Slots to communicate between the threads.
• For basic function (the "Desired" solution above) just follow the existing basic design pattern for PyQt multithreaded GUI applications:
  • A GUI thread whose only job is to display data and relay user inputs / commands, and,
  • A worker thread that does everything else (in this case, including the serial comms).
• One stumbling point along the way: I'd have loved to write the worker thread as one linear flow of code, but unfortunately that's not possible because the worker thread needs to get info from the GUI at times.
  • The only way to get data back and forth between the two threads is via Signals and Slots, and the Slots (i.e. the receiving end) must be a callable, so there was no way for me to implement some type of getdata() operation in the middle of a function. Instead, the worker thread had to be constructed as a bunch of individual functions, each one of which gets kicked off after it receives the appropriate Signal from the GUI.
• Getting the serial data monitoring function (the "Stretch Goal" above) was actually pretty easy -- just have the low-level serial transmit and receive routines already in my code emit Signals for that data, and the GUI thread receives and logs those Signals.

All in all it ended up being a pretty straightforward application of existing principles, but I'm writing it down so hopefully the next guy doesn't have to go down so many blind alleys like I did along the way.