List classes in directory (Python)

Question

I'm developing a Python 2.6 package in which I would like to fetch a list of all classes in a certain directory (within the package) in order to then perform introspection on the class objects.

Specifically, if the directory containing the currently executing module has a sub-dir called 'foobar' and 'foobar' contains .py files specifying class Foo(MyBase), class Bar(MyBase), and class Bar2, I want to obtain a list of references to the class objects that inherit from MyBase, i.e. Foo and Bar, but not Bar2.

I'm not sure if this task actually need involve any dealing with the filesystem or if the modules in the sub-dir are automatically loaded and just need to be listed via introspection somehow. Any ideas here please? Example code is much appreciated, since I'm pretty new to Python, in particular introspection.

Answer 1

Modules are never loaded automatically, but it should be easy to iterate over the modules in the directory and load them with the __import__ builtin function:

import os
from glob import glob
for file in glob(os.path.join(os.path.dirname(os.path.abspath(__file__))), "*.py"):
    name = os.path.splitext(os.path.basename(file))[0]
    # add package prefix to name, if required
    module = __import__(name)
    for member in dir(module):
        # do something with the member named ``member``

Answer 2

Dealt with it myself, this is my version (forked @krakover snippet):

• Iterate directory and import each script placed there
  • Filter out abstract classes
  • Filter out classes that not inherit a base class
  • New instance for each iterated class (change it if you don't find it useful)

---

import importlib
import inspect
import os
import glob


def import_plugins(plugins_package_directory_path, base_class=None, create_instance=True, filter_abstract=True):

    plugins_package_name = os.path.basename(plugins_package_directory_path)

    # -----------------------------
    # Iterate all python files within that directory
    plugin_file_paths = glob.glob(os.path.join(plugins_package_directory_path, "*.py"))
    for plugin_file_path in plugin_file_paths:
        plugin_file_name = os.path.basename(plugin_file_path)

        module_name = os.path.splitext(plugin_file_name)[0]

        if module_name.startswith("__"):
            continue

        # -----------------------------
        # Import python file

        module = importlib.import_module("." + module_name, package=plugins_package_name)

        # -----------------------------
        # Iterate items inside imported python file

        for item in dir(module):
            value = getattr(module, item)
            if not value:
                continue

            if not inspect.isclass(value):
                continue

            if filter_abstract and inspect.isabstract(value):
                continue

            if base_class is not None:
                if type(value) != type(base_class):
                    continue

            # -----------------------------
            # Instantiate / return type (depends on create_instance)

            yield value() if create_instance else value

Usage:

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
plugins_directory_path = os.path.join(SCRIPT_DIR, 'plugins')
plugins = import_plugins(plugins_directory_path, base_class=BasePlugin)

for plugin in plugins:
    plugin.foo()

• imagine there's a sub directory called plugins contains implementations of a BasePlugin class

Answer 3

I wanted to do the same thing, this is what I ended up with:

import glob
import importlib
import inspect
import os

current_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)))
current_module_name = os.path.splitext(os.path.basename(current_dir))[0]
for file in glob.glob(current_dir + "/*.py"):
     name = os.path.splitext(os.path.basename(file))[0]

     # Ignore __ files
     if name.startswith("__"):
         continue
     module = importlib.import_module("." + name,package=current_module_name)

     for member in dir(module):
         handler_class = getattr(module, member)

         if handler_class and inspect.isclass(handler_class):
             print member

Hope it helps..

Answer 4

On platforms that have egrep:

from subprocess import Popen, PIPE
from re import search

def get_classes(directory):
    job = Popen(['egrep', '-ir', '--include=*.py', 'class ', str(directory), ], stdout=PIPE)
    fileout, fileerr = job.communicate()
    if fileerr:
        raise Exception(fileerr)
    while directory[-1] == '/':
        directory = directory[:-1]
    found = []
    for line in fileout.split('\n'):
        match = search('^([^:]+).py:\s*class\s*(\S+)\s*\((\S+)\):', line)
        if match:
            pypath = match.group(1).replace(directory, '').replace('/', '.')[1:]
            cls = match.group(2)
            parents = filter(lambda x: x.strip, match.group(3).split())
            found.append((pypath, cls, parents, ))
    return found

For get_classes('.'), egrep returns something like:

./helpers/action.py:class Action(object):
./helpers/get_classes.py:    job = Popen(['egrep', '-ir', '--include=*.py', 'class ', str(directory), ], stdout=PIPE) # this is the get_classes script; not a valid result
./helpers/options.py:class Option(object):

which is converted into tuples of the path, class name and direct ancestors:

[('helpers.action', 'Action', ['object']), ('helpers.options', 'Option', ['object'])]

If you just want the paths, that's [item[0] for item in get_classes('.')].

Answer 5

Option 1: grep for "^class (\a\w+)\(Myclass" regexp with -r parameter.

Option 2: make the directory a package (create an empty __init__.py file), import it and iterate recursively over its members:

import mymodule
def itermodule(mod):
    for member in dir(mymod):
        ...

itermodule(mymodule)