Python: Implement basic plugin architecture with Python and importlib

plugin_architecture.md

Implementing a basic plugin architecture shouldn't be a complicated task. The solution described here is working but you still have to import every plugin (inheriting from the base class).

This is my solution:

Basic project structure

$ tree
.
├── main.py
└── plugins
    ├── __init__.py
    ├── plugin_a.py
    ├── plugin_b.py

The base plugin

$ cat plugins/__init__.py

import os
import traceback
from importlib import util


class Base:
    """Basic resource class. Concrete resources will inherit from this one
    """
    plugins = []

    # For every class that inherits from the current,
    # the class name will be added to plugins
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.plugins.append(cls)


# Small utility to automatically load modules
def load_module(path):
    name = os.path.split(path)[-1]
    spec = util.spec_from_file_location(name, path)
    module = util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module


# Get current path
path = os.path.abspath(__file__)
dirpath = os.path.dirname(path)

for fname in os.listdir(dirpath):
    # Load only "real modules"
    if not fname.startswith('.') and \
       not fname.startswith('__') and fname.endswith('.py'):
        try:
            load_module(os.path.join(dirpath, fname))
        except Exception:
            traceback.print_exc()

A sample plugin

$ cat plugins/plugin_a.py
import plugins


class PluginA(plugins.Base):

    def __init__(self):
        pass

    def start(self):
        print("Plugin A")

How to use it

$ cat main.py
from plugins import Base

if __name__ == '__main__':
    for p in Base.plugins:
        inst = p()
        inst.start()

Sample run

$ python main.py
Plugin B
Plugin A

Haha, same for me! I've been trying to understand the plugin system for hours on end and find a good method to discover plugins. This is it.

thank you for code

Whoa, this is awesome!

An improvement / simplification from me:

    # Find the plugins folder depending on whether we're packed up using pyinstaller or not
    if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
        plugins_dir = Path(sys._MEIPASS) / "plugins"
    else:
        plugins_dir = Path("plugins")

    for pluginp in plugins_dir.glob("*.py"):
        pstem = pluginp.stem
        if pstem.startswith(".") or pstem.startswith("__"):
            continue
        spec = util.spec_from_file_location(pstem, pluginp)
        module = util.module_from_spec(spec)
        spec.loader.exec_module(module)

    for plugin_cls in PluginBase.plugins:
        plugin = plugin_cls()
        plugin.run()

Is there any way for a plugin to import a module or method in main project?

I'd like to expose some function in main application to plugin codes, but have no idea how to do it.

The main application may be compiled and distributed to user, then user can write their own plugin and call the function exposed by main application to add their specific feature.