Class variables and inheritance in Python

Question

I'd like to be able to define a class variable (in the base class somehow) that is present but not shared among the subclasses or instances. What I initially tried was the following:

from __future__ import print_function  # For lambda print()

class CallList(list):
    def __call__(self, *args, **kwargs):
        for f in self:
            f(*args, **kwargs)

class State:    
    on_enter = CallList()
    def enter(self):
        self.on_enter()

class Opening(State): pass 
class Closing(State): pass

Opening.on_enter.append(lambda: print('Opening state entered'))
Closing.on_enter.append(lambda: print('Closing state entered'))

but the behaviour of the subclasses and subclass instances is to reference the base classes class variable, which gives me the following:

opening = Opening()
closing = Closing()

opening.enter()
# Actual output:  Opening state entered
#                 Closing state entered
# Desired output: Opening state entered

opening.on_enter.append(lambda: print('Additional instance callback'))
opening.enter()
# Actual output:  Opening state entered
#                 Closing state entered
#                 Additional instance callback
# Desired output: Opening state entered
#                 Additional instance callback

closing.enter()
# Actual output:  Opening state entered
#                 Closing state entered
#                 Additional instance callback
# Desired output: Closing state entered

I understand why this is happening (I was expecting something different from experience is other languages, but that's fine).

Is is possible to modify the Base class (and not the subclasses) to get the subclasses to each have their own copy of the class variable, and for the instances of the subclasses to get a copy of their classes variable that can then independently be modified with no sharing as well?

Answer 1

First of all, start using new-style classes by inheriting from object:

class State(object):    
    on_enter = CallList()
    def enter(self):
        self.on_enter()

Old-style classes are old, and what I’m going to suggest won’t work there.

Your specific problem can be solved with descriptors. As you can see in the documentation, descriptors allow us to override attribute access for a specific attribute, namely the attribute they are applied to. This is even possible when the attribute is read from the class.

import weakref

class CallListDescriptor(object):
    def __init__(self):
        self._class_level_values = weakref.WeakKeyDictionary()

    def __get__(self, instance, type):
        if instance is None:
            # class-level access
            try:
                return self._class_level_values[type]
            except KeyError:
                default = CallList()
                self._class_level_values[type] = default
                return default
        # instance-level access
        return instance._call_lists.setdefault(self, CallList())


class State(object):
    def __init__(self):
        # for the instance-level values
        self._call_lists = {}

    on_enter = CallListDescriptor()

We are using weakref for the class-level attributes to ensure that subclasses can get garbage collected properly while the superclass is still in scope.

We can test that it works:

class SubState(State):
    pass

class OtherSubState(State):
    pass

assert State.on_enter is State.on_enter
assert State.on_enter is not SubState.on_enter
assert State.on_enter is not OtherSubState.on_enter
assert SubState.on_enter is SubState.on_enter

instance = SubState()
assert instance.on_enter is not SubState.on_enter

I would however recommend to get rid of the subclass feature and merely ensure instance separate values and then represent state as instances of State instead of subclasses (except if you have a good reason not to, which there perfectly might be):

class CallListDescriptor(object):
    def __get__(self, instance, type):
        if instance is None:
            return self

        return instance._call_lists.setdefault(self, CallList())

class State(object):
    def __init__(self):
        # for the instance-level values
        self._call_lists = {}