metaclass and __prepare__ ()

Question

I am teaching myself about the __prepare__ function. And I see this snippet at PEP3115

# The custom dictionary
class member_table(dict):
    def __init__(self):
        self.member_names = []

    def __setitem__(self, key, value):
        # if the key is not already defined, add to the
        # list of keys.
        if key not in self:
            self.member_names.append(key)

        # Call superclass
        dict.__setitem__(self, key, value)

# The metaclass
class OrderedClass(type):

    # The prepare function
    @classmethod
    def __prepare__(metacls, name, bases): # No keywords in this case
        return member_table()

    # The metaclass invocation
    def __new__(cls, name, bases, classdict):
        # Note that we replace the classdict with a regular
        # dict before passing it to the superclass, so that we
        # don't continue to record member names after the class
        # has been created.
        result = type.__new__(cls, name, bases, dict(classdict))
        result.member_names = classdict.member_names
        return result

class MyClass(metaclass=OrderedClass):
    # method1 goes in array element 0
    def method1(self):
        pass

    # method2 goes in array element 1
    def method2(self):
        pass

My question is at this line: result.member_names = classdict.member_names

How could the variable classdict get a attribute from the member_table class? I see the __prepare__ function returns an instance of member_table, but how is the link between member_table() and classdict.member_names generated?

Many thanks to all of you!

Answer 1

That is pretty straightforward, as it is exactly what prepare does.

3.3.3.3. Preparing the class namespace Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a __prepare__ attribute, it is called as namespace = metaclass.__prepare__(name, bases, **kwds) (where the additional keyword arguments, if any, come from the class definition).

If the metaclass has no __prepare__ attribute, then the class namespace is initialised as an empty ordered mapping.

https://docs.python.org/3/reference/datamodel.html#preparing-the-class-namespace

Which means, the classdict attribute that is passed into the metaclass __new__ and __init__ methods is exactly the same object that is returned by __prepare__.

That object should be a mapping instance, that is, an object that behaves like a dict and have at least the __setitem__ method. This __setitem__ method is called by Python for all variables set inside the the declared class body itself.

That is, for an ordinary class, with no custom metaclass, the variables are recorded in a dictionary (an ordered dictionary, as of Python 3.6).

That happens as Python runs each statement inside the class body. This is the same object that is returned should one call locals() inside the class body:

In [21]: class M(type):
    ...:     @classmethod
    ...:     def __prepare__(cls, *args):
    ...:         class CustomDict(dict):
    ...:             __repr__ = lambda self: "I am a custom dict: " + str(id(self))
    ...:         namespace = CustomDict()
    ...:         print("From __prepare__", namespace)
    ...:         return namespace
    ...: 
    ...:     def __new__(metacls, name, bases, namespace):
    ...:         print("From __new__:", namespace)
    ...:         return super().__new__(metacls, name, bases, namespace)
    ...:     
    ...:     

In [22]: class Test(metaclass=M):
    ...:     def __init__(self):
    ...:         ...
    ...:     print("From class body:", locals(), locals()["__init__"])
    ...:     
    ...:     
From __prepare__ I am a custom dict: 140560887720440
From class body: I am a custom dict: 140560887720440 <function Test.__init__ at 0x7fd6e1bd7158>
From __new__: I am a custom dict: 140560887720440

The main use case when this feature was first designed probably was exactly the possibility of making the order of declarations inside a class body meaningful. That is, a __prepare__ method could just return a collections.OrderedDict instance, and __new__ or __init__ would act upon that order. As of Python 3.6, ordering of class attributes is by default - and the __prepare__ feature remains so advanced one really has to think up of uses for it.

update As of Python (upcoming) 3.12, the mapping returned by __prepare__ and passed in as the fourth argument for the metaclass __new__ method is discarded after the metaclass __init__ method is called. Under PEP 649, approved for Python 3.13, though, a reference to it will be kept so that the lazy annotations mechanisms can retrieve information that might exist only in it.

Edit: Usually __prepare__ is implemented as a class method as it is called before the metaclass is instantiated.

Answer 2

The comment in your __new__ is confusing things about the role of the object returned by __prepare__ and the accepted answer does not clear it up.

The namespace returned by __prepare__ is just a temporary object during class creation that will be passed to __new__, it is not the namespace that your class will use at runtime after creation. (classes have an optimized namespace-object called mappingproxy which enforces keys being strings)

You can see the effect in this script:

class CustomDict(dict):

    def __setitem__(self, key, value):
        print(f"setting {key} to {value}")
        super().__setitem__(key, value)


class PrintAssignmentMeta(type):

    @classmethod
    def __prepare__(mcs, name, bases):
        return CustomDict()
    
    def __new__(mcs, name, bases, namespace):
        return super().__new__(mcs, name, bases, namespace)


class PrintAssignmentClass(metaclass=PrintAssignmentMeta):
    # these print
    a = 3
    b = 4

# this does not print because the class-namespace no longer involves your custom setitem
PrintAssignmentClass.c = 5

# but all three are in the namespace of the class
print(PrintAssignmentClass.a, PrintAssignmentClass.b, PrintAssignmentClass.c)
print(PrintAssignmentClass.__dict__)