CODEWITHSUNDEEP
pythonpython-3.xinheritancesupermetaclass
CaffeinatedMike
CaffeinatedMike

Reputation: 1607

How can I make it a requirement to call super() from within an abstract method of a metaclass?

I have a central metaclass with abstract methods, some of which I would like to not only force them to be present in the child classes, but also want to force the child classes to explicitly call super() from within some of those abstract methods.

Take the following simplified example:

import requests
import abc

class ParentCls(metaclass=abc.ABCMeta):
    # ...
    @abc.abstractmethod
    def login(self, username, password):
        """Override with logic for logging into the service"""

    @abc.abstractmethod
    def download_file(self, resp: requests.Response) -> None:
        # repeated logic goes here
        # basically, save the streaming download either to the cloud 
        # or locally depending on other class variables
        pass

class ChildCls(ParentCls):
    def login(self, username, password):
        # perform the login steps, no need to call `super()` in this method
        pass

    def download_file(self, link):
        # download request will be different per child class
        dl_resp = requests.get(link, stream=True)
        super().download_file(dl_resp)  # how can I make an explicit super() call required?

So, my questions are:

  1. Is it possible to require some (not all) abstract methods to explicitly call super() from the child class?
  2. If yes, how would that look? If not, how come?

Upvotes: 3

Views: 1928

Answers (2)

jsbueno
jsbueno

Reputation: 110311

One difference of a method containing a call to super() and one not is that the first one will have a __class__ nonlocal - variable. That can be inspected by looking at the code object:

In [18]: class A: 
    ...:     def a(self): 
    ...:         super().a() 
    ...:     def b(self): 
    ...:         pass 
    ...:                                                                                                                                                                                     

In [19]: A.a.__code__.co_freevars                                                                                                                                                            
Out[19]: ('__class__',)

In [20]: A.b.__code__.co_freevars                                                                                                                                                            
Out[20]: ()

This is injected by the runtime, before calling the metaclass, as a way for type.__new__ to be able to insert a reference when the class is created to the class itself. It is them used automatically by the parameterless call to super.

The problem is this does not ensure the presence of super(), or that it is actually run, and neither that it is used for calling the desired super-method (one could use it to inspect an attribute, or call another method in the super-class, although that is unusual).

However, this check alone would catch most of the cases, and prevent accidental dismisses.

If you really need to strictly check for the supermethod call, that can be done in runtime, when the actuall method is called, by a much more complicated mechanism:

In pseudo code:

  • Use the metaclass to wrap the leaf-most override of the called method with a decorator (let's call this "client-decorator")
  • decorate the base-methods that must be run with another decorator (say "checker-decorator")
  • The functionality of the "client-decorator" is to flip a flag that should be unflipped by the "checker-decorator".
  • On method-exit, the "client-decorator" can then know if the base method was called, and raise an error if not.

Trying to have a traditional decorator that would encase only the leafmost method in the subclasses is complicated (if you look for my answers on questions tagged metaclass, I've posted example code for that at least once, and there are lots of corner cases). However, it is easier to do if you wrap the method dinamicaly on the class __getattribute__ - when the method is retrieved from the instance.

In this example, both "client-decorator" and "checker-decorator" would have to be coordinated, with access to the "base_method_had_run" flag. One mechanism to do that is for the checker-decorator attach the "client-decorator" to the decorated method, and then the metaclass __new__ method would put up a registry with the method names and client-decorators for each class. This registry can then be used for the dynamic-wrapping by __getattribute__

After writing these paragraphs I have it clear enough I can put up a working example:


from functools import wraps
import abc

def force_super_call(method):
    # If the instance is ever used in parallel code, like in multiple threads
    # or async-tasks, the flag bellow should use a contextvars.ContectVar
    # (or threading.local)
    base_method_called = False
    @wraps(method)
    def checker_wrapper(*args, **kwargs):
        nonlocal base_method_called
        try:
            result = method(*args, **kwargs)
        finally:
            base_method_called = True
        return result

    # This will be used dinamically on each method call:
    def client_decorator(leaf_method):
        @wraps(leaf_method)
        def client_wrapper(*args, **kwargs):
            nonlocal base_method_called
            base_method_called = False
            try:
                result = leaf_method(*args, **kwargs)
            finally:
                if not base_method_called:
                    raise RuntimeError(f"Overriden method '{method.__name__}' did not cause the base method to be called")

                base_method_called = False

            return result
        return client_wrapper

    # attach the client-wrapper to the decorated base method, so that the mechanism
    # in the metaclass can retrieve it:
    checker_wrapper.client_decorator = client_decorator

    # ordinary decorator return
    return checker_wrapper


def forcecall__getattribute__(self, name):

    cls = type(self)

    method = object.__getattribute__(self, name)
    registry = type(cls).forcecall_registry

    for superclass in cls.__mro__[1:]:
        if superclass in registry and name in registry[superclass]:
            # Apply the decorator with ordinary, function-call syntax:
            method = registry[superclass][name](method)
            break
    return method


class ForceBaseCallMeta(abc.ABCMeta):
    forcecall_registry = {}

    def __new__(mcls, name, bases, namespace, **kwargs):
        cls = super().__new__(mcls, name, bases, namespace, **kwargs)
        mcls.forcecall_registry[cls] = {}
        for name, method in cls.__dict__.items():
            if hasattr(method, "client_decorator"):
                mcls.forcecall_registry[cls][name] = method.client_decorator
        cls.__getattribute__ = forcecall__getattribute__
        return cls

And this works:


In [3]: class A(metaclass=ForceBaseCallMeta): 
   ...:     @abc.abstractmethod 
   ...:     @force_super_call 
   ...:     def a(self): 
   ...:         pass 
   ...:          
   ...:     @abc.abstractmethod 
   ...:     @force_super_call 
   ...:     def b(self): 
   ...:         pass 
   ...:                                                                                                                                                                                      

In [4]: class B(A): 
   ...:     def a(self): 
   ...:         return super().a() 
   ...:     def b(self): 
   ...:         return None 
   ...:                                                                                                                                                                                      

In [5]: b = B()                                                                                                                                                                              

In [6]: b.a()                                                                                                                                                                                

In [7]: b.b()                                                                                                                                                                                
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
...
RuntimeError: Overriden method 'b' did not cause the base method to be called

Upvotes: 3

DocDriven
DocDriven

Reputation: 3974

I am not aware of any way you can hook into the Python protocols so that can enforce HOW certain methods have to be implemented. Whoever implements the user side class (e.g. the child) is responsible for the proper implementation of the methods. Documentation is your best friend here.

If you are interested in a workaround, though, please find below an alternative possibility to implement your logic.

import requests
import abc

class ParentCls(metaclass=abc.ABCMeta):
    # ...
    @abc.abstractmethod
    def login(self, username, password):
        """Override with logic for logging into the service"""

    def download_file(self, link) -> None:

        dl_resp = self._get_download_response(link)

        # repeated logic goes here
        # basically, save the streaming download either to the cloud 
        # or locally depending on other class variables
        print('repeated logic')

    @abc.abstractmethod
    def _get_download_response(self, link):
        pass
        
class ChildCls(ParentCls):
    def login(self, username, password):
        # perform the login steps, no need to call `super()` in this method
        pass

    def _get_download_response(self, link):
        # do whatever is child-specific here
        print('construct download request for child')
        return requests.get(link, stream=True)

You can customize your child-specific logic in the respective classes, but the method that you call is only implemented in the parent class.

Upvotes: 2

Related Questions