How to get the original name of a positional parameter in Python?

Question

In Python, is it possible to automagically get the original name of the variables passed to the current function as positional arguments?

It can be convenient for debugging purpose. For example in a pretty-print function to prefix the output of every dumped value with its original name instead of having the caller to manually specify it.

The most obvious way to get something similar would probably be to use keyword arguments like in:

# caution: Python3 code
def pprint_func(**kwargs):
    for n, v in kwargs.items():
        print("{}: {}".format(n, str(v)))

pprint_func(the_name="a value")
# this will output: "the_name: a value"

But this is quite inconvenient as we manually have to name each argument :)

I would rather go for some magic like:

a_var = "Foo"
pprint_func(a_var, some_cat_func("Hello", "World"))

The pprint_func function would find the original name then print it before the dump of the actual value:

a_var: 'Foo'
some_cat_func: 'Hello World'

How can I achieve that in Python3.x?

I assume we need access to the source code of the calling context. Please note that very dirty options like lexing/parsing Python source code manually is definitely a no-go.

Note: this question is related to a previous one of mine. I created a new question because the topic evolved.

Also, I found this answer interesting but it doesn't work if the passed parameter is the direct result of a call (i.e.: pprint_func(result())), resulting in an incomplete solution to my problem. And the final solution offered by the original poster is as dirty as it can be.

Answer 1

PyTest-style import hook

https://docs.pytest.org/en/latest/how-to/writing_plugins.html#pytest.register_assert_rewrite

Instead of lookup for the parameter name when the function gets called, it should be possible to rewrite all calls to dbg_dump to use the keyword argument syntax.

f-strings

https://docs.python.org/3/reference/lexical_analysis.html#f-strings

Python got f-strings in version 3.6. There is a feature specially intended for debugging.

To display both the expression text and its value after evaluation, (useful in debugging), an equal sign '=' may be added after the expression.

By default, the '=' causes the repr() of the expression to be provided, unless there is a format specified. When a format is specified it defaults to the str() of the expression unless a conversion '!r' is declared.

import math
a_var = "Foo"
print(f"{a_var=}, {math.fabs(-1.0)=}")

This prints

a_var='Foo', math.fabs(-1.0)=1.0

It can't be used in the body of dbg_dump, that's too late to resolve the names this way, but it can be useful at the original callsite, to print some such debug f-string instead of calling dbg_dump.

Answer 2

It looks like the only way to do this is to walk through the AST of the caller's context, searching for a call to the current function. Then, thanks to AST's structure, we can easily find every information we need about our positional arguments.

I don't flag this answer as accepted in the hope a Python Guru will pass by and tell the Truth

I've made an implementation that just does the trick and works pretty well for the usual cases. The code definitely deserves some improvements, especially the find_caller_node part since it's as precise as it can be, which is not 100%.

Also I had to read the complete module of the caller since inspect.getsource sometimes didn't return the complete source block (e.g.: the caller was directly located into __main__; Python v3.4.2). Is it a bug or a feature?

Anyway, please don't be too harsh with the code as it is supposed to be used for debugging and educational purpose only.

You can find the latest version here.

And here is a copied/pasted version for the posterity:

#!/usr/bin/env python3
#
# pydump
# A Python3 pretty-printer that also does introspection to detect the original
# name of the passed variables
#
# Jean-Charles Lefebvre <polyvertex@gmail.com>
# Latest version at: http://gist.github.com/polyvertex (pydump)
#
# Usage:
#     dbg_dump(
#         my_var, None, True, 123, "Bar", (4, 5, 6), fcall(), hello="world")
# Result:
#     my_var: 'Foo'
#     None: None
#     Bool: True
#     Num: 123
#     Str: 'Bar'
#     Tuple: (4, 5, 6)
#     fcall(): "Function's Result"
#     hello: 'world'
#

import sys
import pprint
import inspect
import ast

def dbg_dump(
        *args,
        dumpopt_stream=sys.stderr,
        dumpopt_forcename=True,
        dumpopt_pformat={'indent': 2},
        dumpopt_srcinfo=1,
        **kwargs):
    """
    Pretty-format every passed positional and named parameters, in that order,
    prefixed by their **original** name (i.e.: the one used by the caller), or
    by their type name for literals.

    Depends on the *pprint*, *inspect* and *ast* modules, which are part of the
    Python3 standard library.

    Jean-Charles Lefebvre <polyvertex@gmail.com>
    Latest version at: http://gist.github.com/polyvertex (pydump)

    Note that the names of the keyword arguments you want to dump must not start
    with "dumpopt_" since this prefix is used internally to differentiate
    options over values to dump.

    Also, the introspection code won't behave as expected if do recursive calls
    to this function.

    Options can be passed as keyword arguments to tweak behavior and output
    format:
        dumpopt_stream
            May you wish to print() the result directly, you can pass a stream
            object (e.g.: sys.stdout) through this option, that will be given
            to print()'s "file" keyword argument.
            You can also specify None in case you just want the output string
            to be returned without further ado.
        dumpopt_forcename
            A boolean value to indicate wether you want every dumped value to
            be prepended by its name (i.e.: its name or its type).
            If False, only non-literal values will be named.
        dumpopt_forcename
            The dictionary of keyword arguments to give to pprint.pformat()
        dumpopt_srcinfo
            Specify a false value (None, False, zero) to skip caller's info.
            Specify 1 to output caller's line number only.
            Specify 2 to output caller's file name and line number.
            Specify 3 or greater to output caller's file path and line number.

    Example:
        dbg_dump(
            my_var, None, True, 123, "Bar", (4, 5, 6), fcall(), hello="world")
    Result:
        my_var: 'Foo'
        None: None
        Bool: True
        Num: 123
        Str: 'Bar'
        Tuple: (4, 5, 6)
        fcall(): "Function's Result"
        hello: 'world'
    """
    try:
        def _find_caller_node(root_node, func_name, last_lineno):
            # find caller's node by walking down the ast, searching for an
            # ast.Call object named func_name of which the last source line is
            # last_lineno
            found_node = None
            lineno = 0
            def _luke_astwalker(parent):
                nonlocal found_node
                nonlocal lineno
                for child in ast.iter_child_nodes(parent):
                    # break if we passed the last line
                    if hasattr(child, "lineno") and child.lineno:
                        lineno = child.lineno
                    if lineno > last_lineno:
                        break
                    # is it our candidate?
                    if (isinstance(child, ast.Name)
                            and isinstance(parent, ast.Call)
                            and child.id == func_name):
                        found_node = parent
                        break
                    _luke_astwalker(child)
            _luke_astwalker(root_node)
            return found_node

        frame = inspect.currentframe()
        backf = frame.f_back
        this_func_name = frame.f_code.co_name
        #this_func = backf.f_locals.get(
        #    this_func_name, backf.f_globals.get(this_func_name))

        # get the source code of caller's module
        # note that we have to reload the entire module file since the
        # inspect.getsource() function doesn't work in some cases (i.e.:
        # returned source content was incomplete... Why?!).
        # --> is inspect.getsource broken???
        #     source = inspect.getsource(backf.f_code)
        #source = inspect.getsource(backf.f_code)
        with open(backf.f_code.co_filename, "r") as f:
            source = f.read()

        # get the ast node of caller's module
        # we don't need to use ast.increment_lineno() since we've loaded the
        # whole module
        ast_root = ast.parse(source, backf.f_code.co_filename)
        #ast.increment_lineno(ast_root, backf.f_code.co_firstlineno - 1)

        # find caller's ast node
        caller_node = _find_caller_node(ast_root, this_func_name, backf.f_lineno)
        if not caller_node:
            raise Exception("Caller's AST node not found")

        # keep some useful info for later
        src_info = {
            'file': backf.f_code.co_filename,
            'name': (
                backf.f_code.co_filename.replace("\\", "/").rpartition("/")[2]),
            'lineno': caller_node.lineno}

        # if caller's node has been found, we now have the AST of our parameters
        args_names = []
        for arg_node in caller_node.args:
            if isinstance(arg_node, ast.Name):
                args_names.append(arg_node.id)
            elif isinstance(arg_node, ast.Attribute):
                if hasattr(arg_node, "value") and hasattr(arg_node.value, "id"):
                    args_names.append(arg_node.value.id + "." + arg_node.attr)
                else:
                    args_names.append(arg_node.attr)
            elif isinstance(arg_node, ast.Subscript):
                args_names.append(arg_node.value.id + "[]")
            elif (isinstance(arg_node, ast.Call)
                    and hasattr(arg_node, "func")
                    and hasattr(arg_node.func, "id")):
                args_names.append(arg_node.func.id + "()")
            elif dumpopt_forcename:
                if (isinstance(arg_node, ast.NameConstant)
                        and arg_node.value is None):
                    args_names.append("None")
                elif (isinstance(arg_node, ast.NameConstant)
                        and arg_node.value in (False, True)):
                    args_names.append("Bool")
                else:
                    args_names.append(arg_node.__class__.__name__)
            else:
                args_names.append(None)
    except:
        src_info = None
        args_names = [None] * len(args)

    args_count = len(args) + len(kwargs)

    output = ""
    if dumpopt_srcinfo and src_info:
        if dumpopt_srcinfo <= 1:
            fmt = "D({2}):"
        elif dumpopt_srcinfo == 2:
            fmt = "{1}({2}):"
        else:
            fmt = "{0}({2}):"
        output += fmt.format(
            src_info['file'], src_info['name'], src_info['lineno'])
        output += "\n" if args_count > 1 else " "
    else:
        src_info = None

    for name, obj in zip(
            args_names + list(kwargs.keys()),
            list(args) + list(kwargs.values())):
        if name and name.startswith("dumpopt_"):
            continue
        if src_info and args_count > 1:
            output += "  "
        if name:
            output += name + ": "
        output += pprint.pformat(obj, **dumpopt_pformat) + "\n"

    if dumpopt_stream:
        print(output, end="", file=dumpopt_stream)
        return None # explicit is better than implicit
    else:
        return output.rstrip()


if __name__ == "__main__":
    def fcall():
        return "Function's Result"
    my_var = "Foo"
    dbg_dump(
        my_var, None, True, 123, "Bar", (4, 5, 6), fcall(),
        dbg_dump(1, dumpopt_stream=None), hello="world")