Python: Why is __getattr__ catching AttributeErrors?

Question

I'm struggling with __getattr__. I have a complex recursive codebase, where it is important to let exceptions propagate.

class A(object):
    @property
    def a(self):
        raise AttributeError('lala')

    def __getattr__(self, name):     
        print('attr: ', name)
        return 1      

print(A().a)

Results in:

('attr: ', 'a')
1

Why this behaviour? Why is no exception thrown? This behaviour is not documented (__getattr__ documentation). getattr() could just use A.__dict__. Any thoughts?

Answer 1

I know it's been a while since this question was asked, but here is my workaround:

class PropertyAttributeError(Exception):
    pass


def getattr_safe_property(func):
    """The A class uses 'property', and overloads
    :obj:`__getattr__`. This is not a good idea, because if an AttributeError is
    raised during the evaluation of a property, then the error message will
    look like the property itself is not found.

    To avoid those misleading errors, all properties of A must be written as such:


    .. code-block:: python

        @property
        @getattr_safe_property
        def some_property(self):
            ...

    Since the new :obj:`~PropertyAttributeError` is raised from the original
    :obj:`AttributeError`, the error stack will contain the actual problematic
    line, but it will not be silently caught by :obj:`__getattr__`.
    """
    def wrapper(self):
        try:
            return func(self)
        except AttributeError as error:
            raise PropertyAttributeError(
                "An AttributeError was raised while evaluating the property "
                f"'{func.__name__}' of a {self.__class__.__name__} instance:"
                f" {error}"
            ) from error
    return wrapper


class A:

    def __init__(self):
        self._a = 0

    @property
    @getattr_safe_property
    def some_property(self):
        return {}

    def __getattr__(self, item):
        # Do some stuff
        raise AttributeError(f"No attribute {item}")

    @property
    @getattr_safe_property
    def a(self):
        print(self.some_property.wrong_attribute)
        return self._a

    @a.setter
    def a(self, value):
        self._a = value


b = A()
b.a = 3
print(b.a)

You will then get a correct error stack:

Traceback (most recent call last):
  File "path/to/file.py", line 8, in wrapper
    return func(self)
           ^^^^^^^^^^
  File "path/to/file.py", line 34, in a
    print(self.some_property.wrong_attribute)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'wrong_attribute'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "path/to/file.py", line 44, in <module>
    print(b.a)
          ^^^
  File "/path/to/file.py", line 10, in wrapper
    raise PropertyAttributeError(
PropertyAttributeError: An AttributeError was raised while evaluating the property 'a' of a A instance: 'dict' object has no attribute 'wrong_attribute'

Answer 2

Using __getattr__ and properties in the same class is dangerous, because it can lead to errors that are very difficult to debug.

If the getter of a property throws AttributeError, then the AttributeError is silently caught, and __getattr__ is called. Usually, this causes __getattr__ to fail with an exception, but if you are extremely unlucky, it doesn't, and you won't even be able to easily trace the problem back to __getattr__.

EDIT: Example code for this problem can be found in this answer.

Unless your property getter is trivial, you can never be 100% sure it won't throw AttributeError. The exception may be thrown several levels deep.

Here is what you could do:

1. Avoid using properties and __getattr__ in the same class.
2. Add a try ... except block to all property getters that are not trivial
3. Keep property getters simple, so you know they won't throw AttributeError
4. Write your own version of the @property decorator, which catches AttributeError and re-throws it as RuntimeError.

See also http://blog.devork.be/2011/06/using-getattr-and-property_17.html

EDIT: In case anyone is considering solution 4 (which I don't recommend), it can be done like this:

def property_(f):
    def getter(*args, **kwargs):
        try:
            return f(*args, **kwargs)
        except AttributeError as e:
            raise RuntimeError, "Wrapped AttributeError: " + str(e), sys.exc_info()[2]

    return property(getter)

Then use @property_ instead of @property in classes that override __getattr__.

Answer 3

regularly run into this problem because I implement __getattr__ a lot and have lots of @property methods. Here's a decorator I came up with to get a more useful error message:

def replace_attribute_error_with_runtime_error(f):
    @functools.wraps(f)
    def wrapped(*args, **kwargs):
        try:
            return f(*args, **kwargs)
        except AttributeError as e:
            # logging.exception(e)
            raise RuntimeError(
                '{} failed with an AttributeError: {}'.format(f.__name__, e)
            )
    return wrapped

And use it like this:

class C(object):

    def __getattr__(self, name):
        ...

    @property
    @replace_attribute_error_with_runtime_error
    def complicated_property(self):
        ...

    ...

The error message of the underlying exception will include name of the class whose instance raised the underlying AttributeError. You can also log it if you want to.

Answer 4

You're doomed anyways when you combine @property with __getattr__:

class Paradise:
    pass

class Earth:
    @property
    def life(self):
        print('Checking for paradise (just for fun)')
        return Paradise.breasts
    def __getattr__(self, item):
        print("sorry! {} does not exist in Earth".format(item))

earth = Earth()
try:
    print('Life in earth: ' + str(earth.life))
except AttributeError as e:
    print('Exception found!: ' + str(e))

Gives the following output:

Checking for paradise (just for fun)
sorry! life does not exist in Earth
Life in earth: None

When your real problem was with calling Paradise.breasts.

__getattr__ is always called when an AtributeError is risen. The content of the exception is ignored.

The sad thing is that there's no solution to this problem given hasattr(earth, 'life') will return True (just because __getattr__ is defined), but will still be reached by the attribute 'life' as it didn't exist, whereas the real underlying problem is with Paradise.breasts.

My partial solution involves using a try-except in @property blocks which are known to hit upon AttributeError exceptions.

Answer 5

__getattr__ is called when an attribute access fails with an AttributeError. Maybe this is why you think it 'catches' the errors. However, it doesn't, it's Python's attribute access functionality that catches them, and then calls __getattr__.

But __getattr__ itself doesn't catch any errors. If you raise an AttributeError in __getattr__ you get infinite recursion.

Answer 6

__getattribute__ documentation says:

If the class also defines __getattr__(), the latter will not be called unless __getattribute__() either calls it explicitly or raises an AttributeError.

I read this (by inclusio unius est exclusio alterius) as saying that attribute access will call __getattr__ if object.__getattribute__ (which is "called unconditionally to implement attribute accesses") happens to raise AttributeError - whether directly or inside a descriptor __get__ (e.g. a property fget); note that __get__ should "return the (computed) attribute value or raise an AttributeError exception".

As an analogy, operator special methods can raise NotImplementedError whereupon the other operator methods (e.g. __radd__ for __add__) will be tried.

Answer 7

I just changed the code to

class A(object):
    @property
    def a(self):
        print "trying property..."
        raise AttributeError('lala')
    def __getattr__(self, name):     
        print('attr: ', name)
        return 1      

print(A().a)

and, as we see, indeed the property is tried first. But as it claims not to be there (by raising AttributeError), __getattr__() is called as "last resort".

It is not documented clearly, but can maybe be counted under "Called when an attribute lookup has not found the attribute in the usual places".