CODEWITHSUNDEEP
pythonmetaclass
Tanc
Tanc

Reputation: 105

Avoid __new__ to be called every time

I understood that classes are instances of metaclasses and that __new__ is running before __init__, because, you must create an instance before initializing it.

Imagine now the following :

import time
class ConfigurationsMeta(type):
    def __new__(cls, name, bases, attr):
        # Potentially a long task here (eg: Getting value from a web service)
        time.sleep(2)

        # Which class inherit from me (debug)
        print(f'Class {name}')

        config = super().__new__(cls, name, bases, attr)

        #Set a variable to be propagated (Variable coming from web service)
        setattr(config, "URL", "https://stackoverflow.com/")

        return config

class Foo(metaclass=ConfigurationsMeta):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')

class Bar(Foo):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')

class Baz(Bar):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')

e = Foo()
s = Bar()
c = Baz()

  1. Nice because URL is well propagated as I do have 

    Foo : https://stackoverflow.com/
Bar : https://stackoverflow.com/
Baz : https://stackoverflow.com/

  2. I do have now something I don't understand very well :

    Class Foo is written after 2 sec

    Class Bar is written after another 2 sec

    Class Baz is written finally after another 2 sec

So metaclass is executed three times. This must explain that as __new__ is responsible for creating classes, it must be run every time, so three times.

Am I right?

How can I avoid it and make it run only once?

Upvotes: 1

Views: 358

Answers (3)

jsbueno
jsbueno

Reputation: 110271

The other answers cover why you don't need a metaclass here. This answer is to explain briefly what metaclass' __new__ does: it does build your class - it is called by the Python runtime itself when it processes a class <name>(bases, ...): <body> statement. No, you can't have a class without calling a metaclass __new__ method - or at least "the root of all metaclasses", type's __new__ method.

That said, if for some reason you'd need to have a long task that could be run just once for all classes you create, all you'd have to do would be cache the results of the long task, and used the cached values in subsequent calls.

If for some reason the value can not be cached, and must be performed in the metaclass, you'd have to arrange for your classes bodies themselves to be executed in different treads or using an asyncio loop. More elegant forms might include instantiating a concurrent.futures.ThreadPoolExecutor inside the metaclass __new__ when it is called for the first time (and hold it alive as a metaclass attribute), and after calling type.__new__ for each class submit the part that takes long as a future.

As you can see, since what you need is just set a class attribute, you should avoid doing this as a metaclass.

Still, the design could be like:

from concurrent.futures import ThreadPoolExecutor

TIMEOUT = 5

class PostInitParallelMeta(type):
    executor = None
    def __init__(cls, name, bases, ns, **kw):
        super().__init__(name, bases, ns, **kw)
        mcls = cls.__class__
        if not mcls.executor:
             mcls.executor = ThreadPoolExecutor(20)
        mcls.executor.submit(cls._post_init)



class Base(metaclass=PostInitParallelMeta):
    _initalized = False
    def _post_init(cls, url):
        # do long calculation and server access here
        result = ...
        cls.url = result
        cls._initialized = True

    def __init__(self, *args, **kw):
        super.__init__(*args, **kw)
        counter = 0
        while not self.__class__._initialized:
            time.sleep(0.2)
            counter += 0.2
            if counter > TIMEOUT:
                 raise RunTimeError(f"failed to initialize {self.__class__}")


class Foo(Base):
    # set any parameters needed to the initilization task
    # as class attributes
    ...

PS - I just wrote this and realized the code in the metaclass can safely be put in Base's __init_subclass__ method - no need for a metaclass at all even for this.

Upvotes: 0

Pynchia
Pynchia

Reputation: 11590

As mentioned already, you do not need metaclasses.

And you have got an excellent and flexible answer already, which uses __init_subclass__.

A flat simple approach is to set the shared attribute in the superclass and let the instances of the subclasses find it (as they normally do up along the chain of instance, class, superclasses)

class Configurations:
    URL = 'https://stackoverflow.com/'

class Foo(Configurations):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')

class Bar(Foo):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')

class Baz(Bar):
    def __init__(self):
        print(f'{__class__.__name__} : {self.URL}')


e = Foo()
s = Bar()
c = Baz()

Or, if the configuration is more complicated, use a class method to keep the code tidy

class Configurations:

    @classmethod
    def create_cfg(cls):
        cls.URL = 'https://stackoverflow.com/'
...

Configurations.create_cfg()
e = Foo()
s = Bar()
c = Baz()

Both ways initialise the configuration once only and produce

Foo : https://stackoverflow.com/
Bar : https://stackoverflow.com/
Baz : https://stackoverflow.com/

Upvotes: 0

chepner
chepner

Reputation: 531055

You don't really need a metaclass here. Assuming you want URL to be a class attribute, not an instance attribute, you just need to define a base class with a suitable __init_subclass__ definition. The URL should be retrieved first and passed as an argument to __init_subclass__ (via a keyword argument in the class statement).

class Base:
    def __init_subclass__(cls, /, url=None):
        super().__init_subclass__(cls)
        if url is not None:
            cls.URL = url


some_url = call_to_webservice()


class Foo(Base, url=some_url):
    pass


class Bar(Foo):
    pass


class Baz(Bar):
    pass

If URL should be an instance attribute, then replace __init_subclass__ with __init__:

some_url = call_to_webservice()

class Base:
    def __init__(self, /, url):
        self.url = url

class Foo(Base):
    pass

f = Foo(some_url)

Upvotes: 1

Related Questions