Python: Subclass whole hierarchy

Question

My solution is at the bottom of the question, based on MisterMiyagi's example

I wasn't sure how best to phrase the title. My idea is the following. I have an abstract base class with some implementations. Some of these implementations refer to eachother as part of their logic, simplified as follows:

import abc


# the abstract class
class X(abc.ABC):
    @abc.abstractmethod
    def f(self):
        raise NotImplementedError()

# first implementation
class X1(X):
    def f(self):
        return 'X1'

# second implementation, using instance of first implementation
class X2(X):
    def __init__(self):
        self.x1 = X1()

    def f(self):
        return self.x1.f() + 'X2'

# demonstration
x = X2()
print(x.f())  # X1X2
print(x.x1.f())  # X1

Now I want to use these classes somewhere, let's say in another module. However I want to add some extra functionality (for example a function g) to all classes in the hierarchy. I could do that by adding it to the base class X, but I want to keep the functionality defined separately. For example I might want to define the new functionality like this:

class Y(X):
    def g(self):
        return self.f() + 'Y1'

This creates another base class with the new functionality, but of course doesn't add it to the existing implementations X1 and X2. I'd have to use diamond inheritance to get that:

class Y1(X1, Y):
    pass

class Y2(X2, Y):
    pass

# now I can do this:
y = Y2()
print(y.g())  # X1X2Y1

The above works correctly, but there is still a problem. In X2.__init__, an instance of X1 is created. For my idea to work this would have to become Y1 in Y2.__init__. But this is of course not the case:

print(y.x1.g())  # AttributeError: 'X1' object has no attribute 'g'

I think what I might be looking for is a way to turn X into an abstract metaclass, such that its implementations require a 'base' parameter to become classes, which can then be instantiated. This parameter is then used within the class to instantiate other implementations with the correct base.

Creating an instance with the new functionality in the base class would then look something like this:

class Y:
    def g(self):
        return self.f() + 'Y1'

X2(Y)()

Which would result in an object equivalent to an instance of the following class:

class X2_with_Y:
    def __init__(self):
        self.x1 = X1(Y)()

    def f(self):
        return self.x1.f() + 'X2'

    def g(self):
        return self.f() + 'Y1'

However I don't know how to create a metaclass that would do this. I'd like to hear whether a metaclass is the right idea and, if so, how to do it.

Solution

Using MisterMiyagi's example I was able to get something that I think will work. The behaviour is close to the metaclass idea that I had.

import abc


class X(abc.ABC):
    base = object  # default base class

    @classmethod
    def __class_getitem__(cls, base):
        if cls.base == base:
            # makes normal use of class possible
            return cls
        else:
            # construct a new type using the given base class and also remember the attribute for future instantiations
            name = f'{cls.__name__}[{base.__name__}]'
            return type(name, (base, cls), {'base': base})


    @abc.abstractmethod
    def f(self):
        raise NotImplementedError()


class X1(X):
    def f(self):
        return 'X1'


class X2(X):
    def __init__(self):
        # use the attribute here to get the right base for the other subclass
        self.x1 = X1[self.base]()

    def f(self):
        return self.x1.f() + 'X2'


# just the wanted new functionality
class Y(X):
    def g(self):
        return self.f() + 'Y1'

Usage is like this:

# demonstration
y = X2[Y]()
print(y.g())  # X1X2Y1
print(y.x1.g())  # X1Y1
print(type(y))  # 
# little peeve here: why is it not '__main__.X2[Y]'?

# the existing functionality also still works
x = X2()
print(x.f())  # X1X2
print(x.x1.f())  # X1
print(type(x))  #

MisterMiyagi · Accepted Answer

Since you are looking at a way to customise your classes, the easiest approach is to do just that:

class X2(X):
    component_type = X1

    def __init__(self):
        self.x1 = self.component_type()

class Y2(X2, Y):
    component_type = Y1

Since component_type is a class attribute, it allows to specialise different variants (read: subclasses) of the same class.

Note that you can of course use other code to construct such classes. Classmethods can be used to create new, derived classes.

Say that for example your classes are capable of picking the correct subclasses from their hierarchy.

class X2(X):
    hierarchy = X

    @classmethod
    def specialise(cls, hierarchy_base):
        class Foo(cls, hierarchy_base):
            hierarchy = hierarchy_base
        return Foo

Y2 = X2.specialise(Y)

Since Python3.7, it is possible to use __class_getitem__ to write the above as Y2 = X2[Y], similar to how Tuple can be specialised to Tuple[int].

class X2(X):
    hierarchy = X

    def __class_getitem__(cls, hierarchy_base):
        class Foo(cls, hierarchy_base):
            hierarchy = hierarchy_base
        return Foo

Y2 = X2[Y]

Class attributes often serve the functionality of metaclass fields, since they express precisely that. In theory, setting a class attribute is equivalent to adding a metaclass field, then setting it as an attribute for each class. What is exploited here is that Python allows instances to have attributes without classes defining their fields. One can think of class attributes as duck-typing metaclasses.

Python: Subclass whole hierarchy

Answers (2)

Related Questions