How to develop a PySide application plugin?

Question

As I understand it Qt provides 2 main plugin mechanisms:

1. Plugins that extend Qt "Qt Extensions"
2. Plugins that extend applications developed with Qt

I'm interested in developing plugins for my application (2).

I use PySide but can't find any resources about developing application plugins using PySide/PyQt.

Following the C++ Qt documentation I understand that the application has to use the Q_DECLARE_INTERFACE() macro and the plugins have to use both Q_INTERFACES() and Q_EXPORT_PLUGIN2() macros but I don't know the code they represent to try and translate it to python. Or is there another way I've missed?

Update:

The closest thing to a solution I could find so far is Alex Martelli's answer to a similar question. Although it looks like it would work I'd rather use an official Qt approach to avoid any cross-platform issues.

Answer 1

I'm a fan of the Roll Your Own approach.

By a plugin, I mean:

• Plugin: a module or package loaded during runtime which augments or modifies the main module's behavior

I see two requirements for a plugin:

1. Can the main module load the plugin during runtime?
2. Is data accessible between the main module and the plugin?

Assumptions

Developing a plugin system is highly subjective. There are many design decisions to make and there is no One True Way. The usual constraints are time, effort, and experience. Be aware that assumptions must be made and that the implementation often defines terminology (e.g. "plugin" or "package"). Be kind and document those as best as possible.

This plugin implementation assumes:

• A plugin is either a Python file or a directory (i.e. "plugin package")

• A plugin package is a directory with structure:

  plugin_package/
      plugin_package.py  <-- entry point
      other_module.py    <-+
      some_subdir/         |- other files
          icon.png       <-+
  

  Note that a Plugin package isn't necessarily a Python package. Whether or not a plugin package is a Python package depends on how you want to handle imports.

• The plugin name is the same as the plugin entry point's file name, as well as the plugin package directory

• The QApplication's QMainWindow is the primary source of data to share from the main module

1. Loading a plugin module

Two pieces of information are required to load a plugin module: the path to the entry point and the plugin's name. How these are obtained can vary and getting them usually requires string/path parsing. For what follows, assume:

• plugin_path is the absolute path of the plugin entry point and,
• plugin_name is the plugin name (which is the module name by the above assumptions).

Python has implemented and re-implemented the import mechanism numerous times. The import method used at the time of writing (using Python 3.8) is:

import importlib
from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader

loader = SourceFileLoader(plugin_name, plugin_path)
spec = spec_from_loader(plugin_name, loader)
plugin_module = module_from_spec(spec)
spec.loader.exec_module(plugin_module)

# Required for reloading a module
sys.modules[plugin_name] = plugin_module

# # This is how to reload a module
# importlib.reload(plugin_module)

It's probably a good idea to include error handling and a record of which modules have been loaded (e.g. by storing them in a dict). Those details are excluded here for brevity.

2. Sharing data

There are two directions which data may be shared:

• Can the main module access the plugin module's data?
• Can the plugin module access the main module's data?

The main module can access plugin data for free after import (by definition). Simply access the plugin module's __dict__:

# Accessing plugin data from the main module
some_data = plugin_module.__dict__.get('data')

Accessing the main module's data from within the plugin is a trickier problem.

As stated above, I usually consider the QMainWindow as synonymous with the end-user's notion of "the application". It's the primary widget users interact with and, as such, typically houses most of the data or has easy access to it. The challenge is sharing the QMainWindow instance.

The solution to sharing QMainWindow instance data is to make it a singleton. This forces any QMainWindow to be the main window users interact with. There are several ways to create singletons in Python. The two most common ways are probably using metaclasses or modules. I've had the most success using the module singleton approach.

Break the QMainWindow code into a separate Python module. At the module level, create but don't initialize a QMainWindow. Create a module-level instance so that other modules can access the instance as a module attribute. Don't initialize it because init requires a QApplication (and because the main window module is not the application entry point).

# main_window.py
from PySide2 import QtWidgets

class MyMainWindow(QtWidgets.QMainWindow):

    def __init__(self):
        super().__init__()

main_window_instance = MyMainWindow.__new__(MyMainWindow)

Use a separate module, such as main.py, for the application entry point. Import the main window instance, create the QApplication, and initialize the main window.

# main.py
import sys

# Importing the instance makes it a module singleton
from main_window import main_window_instance
from PySide2 import QtWidgets

if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)
    main_window_instance.__init__()
    main_window_instance.show()
    sys.exit(app.exec_())

Importing the main window instance is what makes it a singleton. Creating the QApplication here isn't strictly necessary (it's a singleton, too), but feels cleaner to me.

Now, when plugins are loaded during runtime, they can import the main_window_instance. Because the main_window module has already been loaded by the main.py entry point, it's the main window used by the main module (and not a new instance). Data from the main module can now be accessed from the plugin module.

# plugin.py

# The plugin can now access the main module's data
from main_window import main_window_instance

main_window_instance.setWindowTitle('Changed from plugin')

Comments

The minimum setup requires three files: main.py, main_window.py, and plugin.py. The main_window.py defines and instantiates the main window, the main.py makes it a singleton and initializes it, and the plugin.py imports and uses the instance.

Lots of details were left out in hopes that the primary components could be made apparent. Ideally, the Qt and Python documentation should be enough to fill in the gaps...

There are further considerations, such as how to distribute and manage plugins. Plugins could be hosted remotely, packaged as (zip) archives, bundled as proper Python packages, etc. Plugins can be as simple (or as complex) as you want. Hopefully this answer provides ample inspiration for how you want your plugin system to look.

Answer 2

I think Qt's plugin system is intended to allow people to write C++ plugins compiled as binaries. I don't even know if it's theoretically possible to write plugins in Python that will use a C++ binary interface like that.

If you want to support Python plugins your best bet would be to use one of the many pure python plugin systems out there. I have written a PySide app that uses YAPSY to load plugin scripts. YAPSY is a very simple, compact plugin module. It's very easy to include directly in your app as it's a single file, and is BSD licensed so you can use it commercially. Just search for it on Google. I was even able to package my app using py2exe and still retain the ability to import python source file plugins from a plugin directory.