CODEWITHSUNDEEP
pythoninitmetaclass
sgerbhctim
sgerbhctim

Reputation: 3640

using metaclass for default attribute init

Let's say I have a resource class that looks as such:

class Resource(metaclass=ResourceInit):
    def __init__(self, a, b):
        self.a = a
        self.b = b

My goal is to create a metaclass, i.e. ResourceInit, that handles automatically assigning attributes to the instantiated Resource.

The following code I have, that does not work:

config = {"resources": {"some_resource": {"a": "testA", "b": "testB"}}}

class ResourceInit(type):

    def __call__(self, *args, **kwargs):

        obj = super(ResourceInit, self)

        argspec = inspect.getargspec(obj.__self__.__init__)

        defaults = {}

        for argument in [x for x in argspec.args if x != "self"]:
            for settings in config["resources"].values():
                for key, val in settings.items():
                    if key == argument:
                        defaults.update({argument: val})
            if os.environ.get(argument):
                defaults.update({argument: os.environ[argument]})

        defaults.update(kwargs)
        for key, val in defaults.items():
            setattr(obj, key, val)
        return obj

The idea is, using this metaclass, upon instantiating

res = Resource()

will automatically populate a and b if it exists in the config or as an environment variable..

Obviously this is a dummied example, where a and b will be substantially more specific, i.e. xx__resource__name.

My questions:

  1. Can you pass an argument to the metaclass within the subclass, i.e. resource="some_resource"
  2. How can I get this to work as any normal class if either config or os.environ is not set, i.e. x = Test() results in TypeError: __init__() missing 2 required positional arguments: 'a' and 'b'

Upvotes: 1

Views: 507

Answers (1)

Mad Physicist
Mad Physicist

Reputation: 114548

Alternative

You are making this more complicated than it needs to be. The simple solution looks like this:

def get_configured(name, value, config):
    if value is None:
        try:
            value = next(c for c in config['resources'].values() if name in c)[name]
        except StopIteration:
            value = os.environ.get(name, None)
    return value

class Resource:
    def __init__(self, a=None, b=None):
        self.a = get_configured('a', a, config)
        self.b = get_configured('a', b, config)

The function is reusable, and you can easily modify it to minimize the amount of boilerplate each class will have.

Full Answer

However, if you do insist on taking the metaclass road, you can simplify that as well. You can add as many keyword-only arguments as you want to your class definition (Question 1).

class Resource(metaclass=ResourceInit, config=config): ...

config, and any other arguments besides metaclass will get passed directly to the __call__ method of the meta-metaclass. From there, they get passed to __new__ and __init__ in the metaclass. You must therefore implement __new__. You might be tempted to implement __init__ instead, but object.__init_subclass__, which is called from type.__new__, raises an error if you pass in keyword arguments:

class ResourceInit(type):
    def __new__(meta, name, bases, namespace, *, config, **kwargs):
        cls = super().__new__(meta, name, bases, namespace, **kwargs)

Notice that last arguments, config and kwargs. Positional arguments are passed as bases. kwargs must not contain unexpected arguments before it is passed to type.__new__, but should pass through anything that __init_subclass__ on your class expects.

There is no need to use __self__ when you have direct access to namespace. Keep in mind that this will only update the default if your __init__ method is actually defined. You likely don't want to mess with the parent __init__. To be safe, let's raise an error if __init__ is not present:

        if '__init__' not in namespace or not callable(getattr(cls, '__init__')):
            raise ValueError(f'Class {name} must specify its own __init__ function')
        init = getattr(cls, '__init__')

Now we can build up the default values using a function similar to what I showed above. You have to be careful to avoid setting defaults in the wrong order. So while all keyword-only arguments can have optional defaults, only the positional arguments at the end of the list get them. That means that the loop over positional defaults should start from the end, and should stop immediately as soon as a name with no defaults is found:

def lookup(name, configuration):
    try:
        return next(c for c in configuration['resources'].values() if name in c)[name]
    except StopIteration:
        return os.environ.get(name)

...

        spec = inspect.getfullargspec(init)

        defaults = []
        for name in spec.args[:0:-1]:
            value = lookup(name, config)
            if value is None:
                break
            defaults.append(value)

        kwdefaults = {}
        for name in spec.kwonlyargs:
            value = lookup(name, config)
            if value is not None:
                kwdefaults[name] = value

The expression spec.args[:0:-1] iterates backwards through all the positional arguments except the first one. Remember that self is a conventional, not mandatory name. Removing it by index is therefore much more robust than removing it by name.

The key to making the defaults and kwdefaults values mean anything is to assign them to the __defaults__ and __kwdefaults__ on the actual __init__ function object (Question 2):

        init.__defaults__ = tuple(defaults[::-1])
        init.__kwdefaults__ = kwdefaults
        return cls

__defaults__ must be reversed and converted to a tuple. The former is necessary to get the order of the arguments right. The latter is required by the __defaults__ descriptor.

Quick Test

>>> configR = {"resources": {"some_resource": {"a": "testA", "b": "testB"}}}
>>> class Resource(metaclass=ResourceInit, config=configR):
...     def __init__(self, a, b):
...         self.a = a
...         self.b = b
... 
>>> r = Resource()
>>> r.a
'testA'
>>> r.b
'testB'

TL;DR

def lookup(name, configuration):
    try:
        return next(c for c in configuration['resources'].values() if name in c)[name]
    except StopIteration:
        return os.environ.get(name)

class ResourceInit(type):
    def __new__(meta, name, bases, namespace, **kwargs):
        config = kwargs.pop('config')
        cls = super().__new__(meta, name, bases, namespace, **kwargs)

        if '__init__' not in namespace or not callable(getattr(cls, '__init__')):
            raise ValueError(f'Class {name} must specify its own __init__ function')
        init = getattr(cls, '__init__')
        spec = inspect.getfullargspec(init)

        defaults = []
        for name in spec.args[:0:-1]:
            value = lookup(name, config)
            if value is None:
                break
            defaults.append(value)

        kwdefaults = {}
        for name in spec.kwonlyargs:
            value = lookup(name, config)
            if value is not None:
                kwdefaults[name] = value

        init.__defaults__ = tuple(defaults[::-1])
        init.__kwdefaults__ = kwdefaults

        return cls

Upvotes: 1

Related Questions