Name mangling in function decorators

Question

I cannot use a decorator within a class declared in the same module as the decorator if the decorator name is a __double_leading_underscore type.

It's easier to explain with an example:

# Just a pass-through
def __decorator(fn):
  return fn

decorator = __decorator

class A(object):
  @decorator
  def test(self):
    return 1

print(A().test())
# Prints 1

If I change @decorator with @__decorator:

class A(object):
  @__decorator
  def test(self):
    return 1

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in A
NameError: name '_A__decorator' is not defined

It tries to find __decorator from within the class.

Is there a way to keep the naming convention but refer to the module instead of the class?

Answer 1

As noted in this post it's due to attribute name mangling.

Code posted by OP is an interesting case that made me research how name manging is performed. It turns out that name mangling is performed during compilation to Python bytecode which can be seen by running this code (run in Python 3.7):

import dis

# I want the source code, not just class object.
a_def = '''
class A:
    __mangled = 'aiya!'

    def p(self):
        print(self.__mangled)
'''

print(dis.dis(a_def, depth=2))  # In Python 3.7 they added `depth` argument so nested `code object`s will be printed.

The bytecode is:

  2           0 LOAD_BUILD_CLASS
              2 LOAD_CONST               0 (<code object A at 0x7f4b3f6ddd20, file "<dis>", line 2>)
              4 LOAD_CONST               1 ('A')
              6 MAKE_FUNCTION            0
              8 LOAD_CONST               1 ('A')
             10 CALL_FUNCTION            2
             12 STORE_NAME               0 (A)
             14 LOAD_CONST               2 (None)
             16 RETURN_VALUE

Disassembly of <code object A at 0x7f4b3f6ddd20, file "<dis>", line 2>:
  2           0 LOAD_NAME                0 (__name__)
              2 STORE_NAME               1 (__module__)
              4 LOAD_CONST               0 ('A')
              6 STORE_NAME               2 (__qualname__)

  3           8 LOAD_CONST               1 ('aiya!')
             10 STORE_NAME               3 (_A__mangled)

  5          12 LOAD_CONST               2 (<code object p at 0x7f4b3f6dde40, file "<dis>", line 5>)
             14 LOAD_CONST               3 ('A.p')
             16 MAKE_FUNCTION            0
             18 STORE_NAME               4 (p)
             20 LOAD_CONST               4 (None)
             22 RETURN_VALUE

Disassembly of <code object p at 0x7f4b3f6dde40, file "<dis>", line 5>:
  6           0 LOAD_GLOBAL              0 (print)
              2 LOAD_FAST                0 (self)
              4 LOAD_ATTR                1 (_A__mangled)
              6 CALL_FUNCTION            1
              8 POP_TOP
             10 LOAD_CONST               0 (None)
             12 RETURN_VALUE

This explains why __decorator in class' body triggers searching for _A__decorator because it's "hardcoded" in the bytecode.

The only way to call __decorator in class body is to use one of following calls:

import sys

__mangled = '???'

class A():    
    # All following calls look horrible.
    print(globals()['__mangled'])
    print(eval('__mangled'))
    this_module = sys.modules['__main__']
    print(getattr(this_module, '__mangled'))

As noted each call looks horrible and the issue lies in __mangled name. If your goal is to hint that the module attribute shouldn't be used directly a single underscore is enough. But if you really want leading double underscore you can add more than 2 trailing underscores to prevent mangling as noted in documentation:

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped.

Answer 2

This is due to Python's name mangling. According to the docs

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.

Emphasis added.

When the interpreter sees @__decorator within class A, it ignores the binding to decorator, textually replaces __decorator with _A__decorator and tries to evaluate that identifier, which gives you a NameError.