Tarik
Tarik

Reputation: 81721

__new__ and __init__ in Python

I am learning Python and so far I can tell the things below about __new__ and __init__:

  1. __new__ is for object creation
  2. __init__ is for object initialization
  3. __new__ is invoked before __init__ as __new__ returns a new instance and __init__ invoked afterwards to initialize inner state.
  4. __new__ is good for immutable object as they cannot be changed once they are assigned. So we can return new instance which has new state.
  5. We can use __new__ and __init__ for both mutable object as its inner state can be changed.

But I have another questions now.

  1. When I create a new instance such as a = MyClass("hello","world"), how these arguments are passed? I mean how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.
  2. self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?

I made a little experiments as such below:

>>> class MyClass(tuple):
    def __new__(tuple):
        return [1,2,3]

and I did below:

>>> a = MyClass()
>>> a
[1, 2, 3]

Albeit I said I want to return tuple, this code works fine and returned me [1,2,3]. I knew we were passing the first parameters as the type we wanted to receive once the __new__ function is invoked. We are talking about New function right? I don't know other languages return type other than bound type?

And I did anther things as well:

>>> issubclass(MyClass,list)
False
>>> issubclass(MyClass,tuple)
True
>>> isinstance(a,MyClass)
False
>>> isinstance(a,tuple)
False
>>> isinstance(a,list)
True

I didn't do more experiment because the further wasn't bright and I decided to stop there and decided to ask StackOverflow.

The SO posts I read:

  1. Python object creation
  2. Python's use of __new__ and __init__?

Upvotes: 88

Views: 43872

Answers (3)

Toothpick Anemone
Toothpick Anemone

Reputation: 4644

Your question was something like,

What is the difference between __new__ and __init__ in Python?


Suppose that we have a very simple class named Rectangle

class Rectangle:
    def __init__(self, width:float, height:float, /):
        self._width  = width
        self._hieght = height
        # implicitly return None 

We instantiate the Rectangle class as follows:

rocky_the_rectangle = Rectangle(10.01, 8.91)

The line of code above created a Rectangle having:

width = 10.01
height = 8.91

In order to construct the rectangle, the python interpreter did somthing like the following:

rocky_the_rectangle = Rectangle.__new__(10.01, 8.91)
if isinstance(rocky_the_rectangle, Rectangle):
    Rectangle.__init__(rocky_the_rectangle, 10.01, 8.91)

Note that __init__ can only return None

If you give rocky_the_rectangle to __init__, then __init__ will do the following:

  1. modify the original rocky_the_rectangle. It will not be a copy of rocky.

  2. The output, or return value, from __init__ will be None


The following is true about __new__:

  • __new__ can return somthing other than a new instance of the Rectangle class.

  • __new__ can read and write to attributes of the Rectangle class. The Rectangle class is not directly visible inside of __init__

The description above is most of it. However, if you are still confused, then keep reading.


Below, we have both an __init__ method and a __new__ method.

class Rectangle:
    def __new__(cls, *args, **kwargs, /):
        obj = super().__new__(cls)
        return obj

    def __init__(self, width:float, height:float, /):
        self._width = width
        self._height = height

When we run this code, it can be difficult to see what is going on with our eyes.

There are different ways to debug code, but a quick and easy way is to add some print-statements which help us see the order in which functions get called, and what the input parameters were.

def see_whats_going_on(self_or_cls, *args, **kwargs):
    # Get the string version of the input arguments
    sargs = repr(str(arg))[] for arg in [self_or_cls, *args]

    # Example
    #     sargs = [
    #         "<__main__.Rectangle object at 0x7f7cf24be5d0>",
    #         "10.01",
    #         "8.91"
    #      ]    

    # Now, insert some commas and spaces in-between the args
    comma_seperated_values = ", ".join(sargs)
    csv = comma_seperated_values

    # Example
    #    csv is the following:
    #        "<__main__.Rectangle object at 0x7f7cf24be5d0>, 10.01, 8.91"

    # next, add some parentheses 
    stringed_function_inputs = "(" + csv + ")"

    return stringed_function_inputs 

Let us insert calls to our little print-function into our code:

class Rectangle():
    def __new__(cls, *args, **kwargs):
        see_whats_going_on(cls, *args)        
        obj = super().__new__(cls)
        return obj

    def __init__(self, width:float, height:float, /):
        args = (width, height)
        see_whats_going_on(cls, *args)
        self._width = width
        self._height = height

    @classmethod
    def make_rectangle(cls, *args):
        new_instance = cls.__new__(cls, *args)
        if isinstance(new_instance, cls):
            cls.__init__(new_instance, *args)
        return new_instance

Now we can instantiate the Rectangle class as follows:

rocky_II = Rectangle.make_rectangle(10.01, 8.91)

__init__ allows us to mutate the attributes of self in a modify-in-place manner.

However, __init__ is not allowed to modify self by passing self into a function which returns a modified copy of self.

# NOT ALLOWED INSIDE OF __init__
new_self = paint_it_blue(old_self)

The __init__ method is unable to replace the self parameter with something new and different.

For example, people sometimes want to put a wrapper around the self parameter. You might want something like this:

import functools
import sys

class decorator:

    def __new__(cls, kallable):
        instance = super().__new__(cls)
        instance = functools.update_wrapper(instance, kallable)
        return instance

    def __init__(self, kallable):
        self._kallable = kallable
        self._file     = sys.stdout

    def __call__(self, *args, **kwargs):
        print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")", file=self._file)
        return self._kallable(*args, **kwargs)

@decorator
def pow(base:float, exp:int):
    """
         +------------------------------------------+
         |                EXAMPLES                  |
         +------------------------------------------+
         | BASE | EXPONENT |       OUTPUT           |
         +------+----------+------------------------+
         |    2 |        5 | 2^5      |          32 |
         |  2.5 |        7 | 2.5^7    | 610.3515625 |
         |   10 |        3 | 10^3     |        1000 |
         |  0.1 |        5 | 0.1^5    |     0.00001 |
         |    7 |        0 | 7^0      |           1 |
         +------+----------+----------+-------------+
    """
    base = float(base)
    # convert `exp` to string to avoid flooring, or truncating, floats
    exp  = int(str(exp))
    if exp > 0:
        return base * pow(base, exp-1)
    else: # exp == 2
        return 1

result1 = pow(2, 5)
result2 = pow(8.1, 0)
print("pow(2, 5) == "  , result1)
print("pow(8.1, 0) == ", result2)

print("What is the documentation string? The doc-string is... ", pow.__doc__)

It will not help to use functools.update_wrapper inside of __init__. If you try to write the following...

class f:
    def __init__(outter, inner):
        # `outter` is usually named `self`, but you are...
        #     ... allowed to use other names for it.  
        outter = functools.update_wrapper(outter, inner) 

... then outter will get ignored. You cannot replace the parameter named self with a different self

__new__ allows us to replace self with a wrapper around self

class decorator:

    def __new__(cls, kallable):
        instance = super().__new__(cls)
        instance = functools.update_wrapper(instance, kallable)
        return instance

Without using functools.update_wrapper inside of __new__ the doc-string inside of the original callable will be wiped-out, ignored, not inherited, and/or shadowed.

Upvotes: -1

Demis the Anacronist
Demis the Anacronist

Reputation: 121

seems that nobody had cover how the arguments are handled through the instantiation

lets build a simple class that define new and init

censured = ["Cicciogamer"]

class Foo(object):
    def __new__(cls, name):
        if name in censured:
            print("you shouldn't do this")
        return super().__new__(cls)

    def __init__(self, var):
        self.var = var

when you call a class object to get an instance, python implicity call

Foo.__call__(*args, **kwargs)

so with the class above you get the (maybe) undesired double arguments passing:

foo = Foo("Cicciogamer")
>>> "you shouldn't do this"
foo.var
>>> "Cicciogamer"
    

To achieve the control on how these parameters behave ASFAIK you must override the call method of the CLASS OBJECT not instance

you can use metaclasses but it might be overkill

class MetaFoo:
    def __call__(cls, name, var):
        foo_object = cls.__new__(name)
        cls.__init__(foo_object, var)
        return foo_object

class Foo(metaclass=MetaFoo):
    ...


foo = Foo("Cicciogamer", 505)
>>> "you shouldn't do this"
foo.var
>>> 505

or i think you can achieve this with simply

class Foo:
    ...

    @classmethod
    def new(cls, name, var):
        foo_object = cls.__new__(name)
        cls.__init__(foo_object, var)
        return foo_object

    __call__ = new

please let me notice if this could be achieved in a better way, or if i simply screwed up something

Upvotes: 3

Fred Foo
Fred Foo

Reputation: 363577

how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.

Only rarely will you have to worry about __new__. Usually, you'll just define __init__ and let the default __new__ pass the constructor arguments to it.

self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?

Both are just parameter names with no special meaning in the language. But their use is a very strong convention in the Python community; most Pythonistas will never change the names self and cls in these contexts and will be confused when someone else does.

Note that your use of def __new__(tuple) re-binds the name tuple inside the constructor function. When actually implementing __new__, you'll want to do it as

def __new__(cls, *args, **kwargs):
    # do allocation to get an object, say, obj
    return obj

Albeit I said I want to return tuple, this code works fine and returned me [1,2,3].

MyClass() will have the value that __new__ returns. There's no implicit type checking in Python; it's the responsibility of the programmer to return the correct type ("we're all consenting adults here"). Being able to return a different type than requested can be useful for implementing factories: you can return a subclass of the type requested.

This also explains the issubclass/isinstance behavior you observe: the subclass relationship follows from your use of class MyClass(tuple), the isinstance reflects that you return the "wrong" type from __new__.

For reference, check out the requirements for __new__ in the Python Language Reference.

Edit: ok, here's an example of potentially useful use of __new__. The class Eel keeps track of how many eels are alive in the process and refuses to allocate if this exceeds some maximum.

class Eel(object):
    MAX_EELS = 20
    n_eels = 0

    def __new__(cls, *args, **kwargs):
        if cls.n_eels == cls.MAX_EELS:
            raise HovercraftFull()

        obj = super(Eel, cls).__new__(cls)
        cls.n_eels += 1
        return obj

    def __init__(self, voltage):
        self.voltage = voltage

    def __del__(self):
        type(self).n_eels -= 1

    def electric(self):
        """Is this an electric eel?"""
        return self.voltage > 0

Mind you, there are smarter ways to accomplish this behavior.

Upvotes: 72

Related Questions