Python class inheritance - spooky action

Question

I've observed a strange effect with class inheritance. For the project I'm working on, I'm creating a class to act as a wrapper to another module's class.

I am using the 3rd-party aeidon module (used for manipulating subtitle files) but the issue is probably less specific.

Here is how you'd normally utilize the module...

project = aeidon.Project()
project.open_main(path)

Here is the example 'wrapper' class in use (of course the real class has many methods):

class Wrapper(aeidon.Project):
    pass

project = Wrapper()
project.open_main(path)

This aforementioned code raises an AttributeError upon execution. However, the following works as I originally expected it to:

junk = aeidon.Project()
project = Wrapper()
project.open_main(path)

I named this question after spooky action at a distance because I suspect that it involves global vars/objects in the environment, but I don't know.

I ended up using composition to solve this problem (i.e. self.project = aeidon.Project()) but I'm still curious about this. Can anyone explain what's going on here?

Here is the traceback:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
 in ()
----> 1 project.open_main(path)

/usr/lib/python3/dist-packages/aeidon/deco.py in wrapper(*args, **kwargs)
    208     def wrapper(*args, **kwargs):
    209         frozen = args[0].freeze_notify()
--> 210         try: return function(*args, **kwargs)
    211         finally: args[0].thaw_notify(frozen)
    212     return wrapper

/usr/lib/python3/dist-packages/aeidon/agents/open.py in open_main(self, path, encoding)
    161         format = aeidon.util.detect_format(path, encoding)
    162         self.main_file = aeidon.files.new(format, path, encoding)
--> 163         subtitles = self._read_file(self.main_file)
    164         self.subtitles, sort_count = self._sort_subtitles(subtitles)
    165         self.set_framerate(self.framerate, register=None)

/usr/lib/python3/dist-packages/aeidon/project.py in __getattr__(self, name)
    116             return self._delegations[name]
    117         except KeyError:
--> 118             raise AttributeError
    119 
    120     def __init__(self, framerate=None, undo_limit=None):

AttributeError:

I've tried it both with and without a call to Project's __init__(). Obviously this is not really something that should be done under normal circumstances, I'm just perplexed why Wrapper() would function as expected only after creating a junk aeidon.Project().

Martijn Pieters · Accepted Answer

The aedion.project module does two things:

It adds methods from classes in the aedion.agents package to the class, in order for the documentation generator to include these when extracting docstrings and other information, using a ProjectMeta metaclass:

class ProjectMeta(type):

    """
    Project metaclass with delegated methods added.
    Public methods are added to the class dictionary during :meth:`__new__`
    in order to fool Sphinx (and perhaps other API documentation generators)
    into thinking that the resulting instantiated class actually contains those
    methods, which it does not since the methods are removed during
    :meth:`Project.__init__`.
    """

these methods, if used, would not be correctly bound however.

The Project.__init__ method calls Project._init_delegations(). This method deletes the delegated methods from the class:
```
# Remove class-level function added by ProjectMeta.
if hasattr(self.__class__, attr_name):
    delattr(self.__class__, attr_name)
```
Note the use of self.__class__ here. This is needed because Python won't look for the delegated method via the __getattr__ hook if the method is found on the class instead.

The delegated methods are bound to a dedicated agent instance, so are in fact delegating to that agent:
```
agent = getattr(aeidon.agents, agent_class_name)(self)
# ...
attr_value = getattr(agent, attr_name)
# ...
self._delegations[attr_name] = attr_value
```

When you create a wrapper around this class, the deletion step fails. self.__class__ is your wrapper, not the base Project class. Thus the methods are bound incorrectly; the metaclass-provided methods are being bound, and the __getattr__ hook is never invoked to find the delegated methods instead:

>>> import aeidon
>>> class Wrapper(aeidon.Project): pass
... 
>>> wrapper = Wrapper()
>>> wrapper.open_main
>
>>> wrapper.open_main.__self__
<__main__.Wrapper object at 0x1106313a8>
>>> wrapper._delegations['open_main']
>
>>> wrapper._delegations['open_main'].__self__

because the open_main method on Project still exists:

>>> Project.open_main

As soon as you create an instance of Project those methods are deleted from the class:

>>> Project()

>>> Project.open_main
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: type object 'Project' has no attribute 'open_main'

and your wrapper would start to work as now the delegated open_main is found:

>>> wrapper.open_main
>
>>> wrapper.open_main.__self__

Your wrapper will have to do the deletions itself:

class Wrapper(aeidon.Project):
    def __init__(self):
        super().__init__()
        for name in self._delegations:
            if hasattr(aeidon.Project, name):
                delattr(aeidon.Project, name)

Note that if the aedion maintainers replaced self.__class__ with just __class__ (no self) their code would still work and your subclass approach would also work without having to manually do the class clean-up again. That's because in Python 3, the __class__ reference in methods is an automatic closure variable pointing to the class on which a method was defined. For Project._init_delegations() that'd be Project. Perhaps you could file a bug report to that effect.

Python class inheritance - spooky action

Answers (1)

Related Questions