What does `super()` mean in `__new__`

Question

Note: Part of a flyweight implementation with Python

import weakref

class CarModel:
    _models = weakref.WeakValueDictionary()

    def __new__(cls, model_name, *args, **kwargs):
        model = cls._models.get(model_name)
        if not model:
            model = super().__new__(cls)
            cls._models[model_name] = model    
        return model

    def __init__(self, model_name, air=False):
        if not hasattr(self, "initted"):
          self.model_name = model_name
          self.air = air
          self.initted=True

Question 1> what does super() mean? Does it mean the parent class of CarModel?

Question 2> I also have difficulties to understand the how the function __new__ works? Specifically, the following line.

model = super().__new__(cls)

Description for __new__:

The constructor function is called __new__ as opposed to __init__, and accepts exactly one argument, the class that is being constructed (it is called before the object is constructed, so there is no self argument). It also has to return the newly created object.

Answer 1

I think part of this answer is confused about the relationship between __new__ and __init__. I was confused about this myself. Here's how to answer these kind of questions.

Before we begin it is important to note that defining __new__ always has an invisible/magic @classmethod decorator applied to it.

from inspect import signature # find function signature

class JustInit(object):
  def __init__(self, a):
    self.a = a

print("JustInit")
print(signature(JustInit.__new__))
print(signature(JustInit.__init__))

JustInit
(*args, **kwargs)
(self, a, b)

Well, that's not helpful. Let's inspect those *args and **kwargs by printing them and passing them through with super as suggested by @Dolda2000

class PrintNew(object):
  def __new__(cls, *args, **kwargs):
    print("new args:", args, "new kwargs:", kwargs)
    return super().__new__(cls, *args, **kwargs)

  def __init__(self, a):
    self.a = a

print("PrintNew")
print(signature(PrintNew.__new__))
print(signature(PrintNew.__init__))

pnew = PrintNew(42)
print(pnew.a)

Results in:

PrintNew
(cls, *args, **kwargs)
(self, a)
new args: (42,) new kwargs: {}
Traceback (most recent call last):
  File "/home/rett/notes/python/super_new.py", line 21, in <module>
    pnew = PrintNew(42)
  File "/home/rett/notes/python/super_new.py", line 12, in __new__
    return super().__new__(cls, *args, **kwargs)
TypeError: object.__new__() takes exactly one argument (the type to instantiate)

Wait... I thought the signature of new was *args, **kwargs... oh, the *args must really be just the cls?

While this is true, it is extremely important to point out that:

1. object only takes the cls variable (it doesn't really take *args/**kwargs)
2. your implementation of __new__ can take whatever values are passed to your __init__ and do whatever you want with them.
3. it seems like maybe __new__ returns a non-instantiated class, which then gets __init__ called on it.

With these theories, let's try again by replacing return super().__new__(cls, *args, **kwargs) with return super().__new__(cls)

It now behaves as expected:

PrintNew
(cls, *args, **kwargs)
(self, a)
new args: (42,) new kwargs: {}
42

This might give you some ideas on how you can do your implementation.

Answer 2

I believe, you are using Python3, where the super does not need to be provided with the same class name. Super refers to the Base classes of the current class and does the proper Method Resolution Order for you to invoke the method from the correct base class. __new__ is the method which is invoked to create an instance. It is the first step in the instance creation.

Answer 3

To begin with super() in itself is simply shorthand for super(A, B), where A is the class wherein the code occurs, and B is the first argument to the function in which the code occurs; so in your particular case, super().__new__(cls) expands to super(CarModel, cls).__new__(cls).

In turn, super(T, O) returns a "super object". To understand what a super object does, you need to understand how attribute references on instances and classes work in Python.

Assuming no __getattr__ or __getattribute__ methods are involved, referencing attribute A on an object O (that is, evaluating O.A or getattr(O, "A")) proceeds through the following steps:

1. If "A" is defined in O's instance dict (O.__dict__), then the value on that dict is returned directly, precisely as it is.
2. Otherwise, each of the classes in O's method resolution order are checked in turn, looking for "A" in each of their dicts. If found, call the value D.
3. If D, in turn, does not define a __get__, then it is returned as it is. If it does, however, then D is referred to as a "descriptor", and its __get__ method is called with O as the first argument, and type(O) as the second argument.

An attribute reference on a class works about the same, substituting the class being reference for the instance, with the following differences:

• Step 1 doesn't apply.
• The __get__ method is called with None as the first argument, and the class being referenced as the second.

Python uses descriptors to implement such things as instance methods, class methods, static methods, and properties.

A super object created with super(T, O), then, is a (built-in) object with a __getattribute__ method which is called on every attribute reference on it, and looks up the attribute in the dicts of the only classes following T in O's MRO. The value it then finds, it calls __get__ on as usual.

The process is a bit complex, so as an example, here's how it would work on your specific case. Since CarModel is defined as it is, its MRO is [CarModel, object].

1. super().__new__(cls) expands to super(CarModel, cls).__new__(cls), as described above.
2. super(CarModel, cls) is evaluated to yield a super object S.
3. Python fetches the attribute "__new__" on S (the equivalent of calling getattr(S, "__new__") in Python code).
4. Since S was created on the CarModel class, it considers the classes following CarModel in the MRO of CarModel, and finds "__new__" in the dict of the object class itself. Its value, a static method, has a __get__ method, which is called with the arguments None and cls. Since __new__ is a static method, its __get__ method simply returns the function as it is, unmodified. Therefore, super(CarModel, cls).__new__ is precisely the same as object.__new__.
5. The function obtained in the last step (that is, object.__new__) is called with the cls argument, where cls is probably CarModel, finally a new instance of the CarModel class.

I hope that was at all understandable.

(For the sake of completeness, it should be mentioned that the actual __new__ function on the object class is not actually a static method, but a special built-in function that simply has no __get__ method at all, but since the __get__ method on static methods just return the function they were defined with, the effect is the same.)

Answer 4

super() is used to reference the superclass (i.e. the parent class from which you are subclassing).

__new__ is a method that is invoked to create a new instance of the class, if it is defined.