Append to a member list when subclassing in Python

Question

Say I have this class:

class FooClass(object):
    foos = ["foo", "bar"]

    def do_foos(self):
        for foo in self.foos:
            print("I have a " + foo)

    # ...

I want to create a SpecialisedFooClass for this class which extends FooClass, adding an item "spec" to foos (i.e. so that foos contains ["foo", "bar", "spec"]).

SpecialisedFooClass.foos should depend on FooClass.foos: If I change FooClass' definition so that foos contains ["foo", "bam", "bat"], SpecialisedFooClass.foos should then contain ["foo", "bam", "bat", "spec"].

---

This is the best way I've come up with so far:

class SpecialisedFooClass(FooClass):
    foos = FooClass.foos + ["spec"]

But I find the explicit reference to FooClass concerning. When I decide to add an intermediary subclass (i.e. when SpecialisedFooClass' superclass changes) I will inevitably forget to update this reference. I have in fact already made this mistake IRL with the codebase I'm working on (which doesn't actually deal with foos, bams, and bats...).

---

There's actually no special requirement in my case that foos is a class member rather than an instance member, so this would also work, but I find it ugly. Also, the super call still has an explicit class reference - less worrying here because its to the class it appears in, though.

class FooClass(object):
    def __init__(self, *args, **kwargs):
        self.foos = ["foo", "bar"]

    def do_foos(self):
        for foo in self.foos:
            print("I have a " + foo)

class SpecialisedFooClass(FooClass):
    def __init__(self, *args, **kwargs):
        super(SpecialisedFooClass, self).__init__(*args, **kwargs)
        self.foos.append("spec")

---

What other options exist, and is there a "Pythonic" way to do this?

Answer 1

Using the fantastic answer about metaclasses, take Django Rest Framework as an example. Every time we want to use permission_classes in theory works but when you go unit testing it, you can see that indeed we are overriding it.

Take the IsAuthenticated permission. If you use in a browser, it will block from accessing APIs if is not logged in but if we want to add an extra permission class, for instance, for Staff on an API level, unit testing it will fail if you test needs to be logged in to access StaffAPI. Using the great metaclass example, we can change thatr behavior to be like

class BaseAuthMeta(type):
    """Metaclass to create/read from permissions property
    """

    def __new__(cls, name, bases, attrs):
        permissions = []
        for base in bases:
            if hasattr(base, 'permissions'):
                permissions.extend(base.permissions)
        attrs['permissions'] = permissions + attrs.get('permissions', [])
        return type.__new__(cls, name, bases, attrs)

Then we just need to make sure we use the property permissions on an API level but we declare in the base of all APIs (if you so wish)

class AccessMixin(metaclass=BaseAuthMeta):
    """
    Django rest framework doesn't append permission_classes on inherited models which can cause issues when
    it comes to call an API programmatically, this way we create a metaclass that will read from a property custom
    from our subclasses and will append to the default `permission_classes` on the subclasses of AccessMixin
    """
    pass


class MyAuthMixin(AccessMixin, APIView):
    """
    Base APIView requiring login credentials to access it from the inside of the platform
    Or via request (if known)
    """
    permissions = [IsAuthenticated]

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.permission_classes = self.permissions

And on an API level

class MyStaffApiView(MyAuthMixin, APIView):
    permissions = [MyCustomStaffPermission]
    ...

If you debug it and if you do a check on self.permission_classes this will actually append what you need (for permissions) since you are using your new custom class attribute.

Sorry for being so late but the metaclass answer is indeed fantastic :)

Answer 2

This might get you close:

class A(object):
    foo_ = ['a']

    @classmethod
    def foo(cls): 
        return sum((getattr(c, 'foo_', []) for c in cls.__mro__[::-1]), [])

class B(A):
    foo_ = ['b']

class C(B):
    foo_ = ['c','d','e']

class D(A):
    foo_ = ['f', 'g', 'h']

class E(B,D):
    foo_ = ['i', 'j', 'k']


for cls in (A,B,C,D,E):
    print cls.__name__, cls.foo()

prints

A ['a']
B ['a', 'b']
C ['a', 'b', 'c', 'd', 'e']
D ['a', 'f', 'g', 'h']
E ['a', 'f', 'g', 'h', 'b', 'i', 'j', 'k']

EDIT

Converted to a classmethod so it is not necessary to instantiate to get the foo list. Also added some diamond inheritance to show this would look.

Answer 3

There are a few ways you can handle this. One option, when attributes require some amount of computation, is to convert it to a property

class FooClass(object):
    foos = ["foo", "bar"]


class SpecializedFooClass(FooClass)

    @property
    def foos(self):
        return super(SpecializedFooClass, self).foos + ['spec']

You can do some optimization so that it's only computed once, or so that it can be changed at runtime instead of returning the same thing every time.

class SpecializedFooClass(FooClass)

    def __init__(self):
        super(SpecializedFoo, self).__init__()
        self._foos = None

    @property
    def foos(self):
        if self._foos is None:
            self._foos = super(SpecializedFooClass, self).foos + ['spec']
        return self._foos

The major downside of using a property is this context is that it won't really behave like a class attribute anymore since you'll have to instantiate a class to get the value.

You can also use metaclasses (great answer on metaclasses). In some cases, this can drastically reduce your code base, but it can also be confusing if you have a very deep inheritance chain and it's not clear that a metaclass is being used. But on the plus side, it will work exactly like a class attribute because it actually is a class attribute.

class FooMeta(type):

    def __new__(cls, name, bases, attrs):
        foos = []
        for base in bases:
            if hasattr(base, 'foos'):
                foos.extend(base.foos)
        attrs['foos'] = foos + attrs.get('foos', [])
        return type.__new__(cls, name, bases, attrs)


class Foo(object):
    __metaclass__ = FooMeta
    foos = ['one', 'two']


class SpecialFoo(Foo):
    foos = ['three']

print Foo.foos
# ['one', 'two']

print SpecialFoo.foos
# ['one', 'two', 'three']

A third option is using class decorators. It's a little less magic than metaclasses, but it also means you're going to have to remember to decorate each subclass. It will function exactly like a class attribute because it actually is one.

def set_foos(cls):
    foos = []
    for base in cls.__bases__:
        if hasattr(base, 'foos'):
            foos.extend(base.foos)
    cls.foos = foos + getattr(cls, 'foos', [])
    return cls


class FooClass(object):
    foos = ['foo', 'bar']


@set_foo
class SpecialFoo(FooClass):
    foos = ['spec']