How to declare names of variables in module?

Question

Motivation

I have following motivational problem - I want use a slightly boosted logging in my project. For that purpose, I am creating module my_logging with similar usage as logging. Most importantly, my_logging needs to have methods debug, info, etc.

Question

Suppose you have a bunch of methods method_1, method_2, .. you want to have in a module module (e.g. debug, info, ... in my_logging) and you know that implementation of these methods will be fairly simmilar.

What is the cleanest way to implement this?

Possible solutions

Solution 1

Define each method separetely, using parametrized method. You are able to do this, since you know all the methods in advance.

def _parametrized_general_method(params):
    ...

def method_1():
    _parametrized_gereral_method(method_1_params)

def method_2():
    _parametrized_gereral_method(method_2_params)
...

Discussion

Obviously, this is too lenghty and there is too much of repeated code. It is tedious, if there is too much of these methods.

On the other hand, methods are declared at 'compile time' and it works well with typing and so on.

The tedium can be much avoid by generating the code.

METHODS = [
    ('method_1', method_1_params), 
    ('method_2', method_2_params),
    ...
]

method_template = '''
def {0}():
    _parametrized_gereral_method({1})
    
'''

with open('module.py', 'w') as f:
    # Probably some module header here
    for name, params in METHODS:
        print(
            method_template.format(name, params)
            file=f
        )
    # Maybe some footer

But this forces you to care about python file, which does not directly belong to your project. You need to have the genarator file in case you want to do some changes in module and run the file. This does not belong (in my opinion) to standard developing cycle and therefore it is not very nice, although very effective.

Also, for sake of completeness, instead of _parametrized_gereral_method you can have something as method_factory in following snippet.

def method_factory(method_params):
    def _parametrized_general_method(params):
        ...
    return _parametrized_general_method(method_params)

method_1 = method_factory(method_1_params)
method_2 = method_factory(method_2_params)

Solution 2

More cleaner way from my point of view it to create these methods at runtime.

METHODS = [
    ('method_1', method_1_params), 
    ('method_2', method_2_params),
    ...
]

for name, params in METHODS:
    globals()[name] = method_factory(params)

Discussion

I consider this to be very elegent way and from purely Pythonic 'dynamic' view as Ok.

Problem arrives with IDEs and their help in form of reference resolution and typing at 'compile time'.

import module

module.method_1()

If you use module from another module, the methods are not found, of course and a warning appears (at 'compile time', it is not actual error). In PyCharm

Cannot find reference 'method' in 'module.py'

Obviously, you don't want to globally supress these warnings, as they are usually very helpful. Moreover, such warnings are one of the reasons why to use IDE.

Also, you can supress it for a particular line (as any warning in PyCharm), but that is way to much pollution in the code.

Or you can ignore the warnings which is very, very bad habit in general.

Solution 3 - maybe?

In Python module, you are able to specify what names the partical module exports / provides by attribute __all__.

In ideal world something like this works.

METHODS = (
    ('method_1', method_1_params), 
    ('method_2', method_2_params),
    ...
)

__all__ = [name for name, params in METHODS]

for name, params in METHODS:
    globals()[name] = method_factory(params)

Discussion

See that __all__ can be evaluated at 'compile time', as METHODS is not used anywhere before assingment of __all__.

My idea is that this will properly notify other modules about names of not-yet-created methods and no warning will appear while the nice dynamic creation of functions is preserved.

Problem is that it does not work as planned because apparently Python cannot regoznize such all attribute and reference warnings in importing modules are still present.

More specific question

Is there a way how to make this approach with __all__ work? Or is there something in similar fashion?

Thank you.

Answer 1

For IDE-level solution - Refer PyCharm document.

Someone extracted list of available suppressions, found one there:

import module

# noinspection PyUnresolvedReferences
module.a()

module.b()

Will only disable inspection one line.

---

Alternatively, if all you wanted is not writing multiple similar functions by yourself - you could just make python do it for you:

from os import path


function_template = '''
def {0}():
    print({0})
    
'''

with open(path.abspath(__file__), 'a') as fp:
    for name, arg in zip('abcde', range(5)):
        fp.write(function_template.format(name, arg))

This will extend current script with generated functions.

---

Elif you just want to wrap logging functions with least typing effort, try closure.

import logging


def make_function(name: str = None):
    logger = logging.getLogger(name)
    logging.basicConfig(format="%(asctime)s | %(name)s | %(levelname)s - %(message)s")

    def wrapper(log_level):
        level_func = getattr(logger, log_level)

        def alternative_logging(msg, *args):
            nonlocal level_func
            level_func(msg)
            # add some actions here, maybe with args

        return alternative_logging

    return map(wrapper, ('debug', 'info', 'warning', 'error', 'critical'))


debug, info, warning, error, critical = make_function('Nice name')
debug2, info2, warning2, error2, critical2 = make_function('Bad name')


warning('oh no')
warning2('what is it')
error('hold on')
error2('are ya ok')
critical('ded')
critical2('not big surprise')

Output:

2020-09-06 12:06:59,742 | Nice name | WARNING - oh no
2020-09-06 12:06:59,742 | Bad name | WARNING - what is it
2020-09-06 12:06:59,742 | Nice name | ERROR - hold on
2020-09-06 12:06:59,742 | Bad name | ERROR - are ya ok
2020-09-06 12:06:59,742 | Nice name | CRITICAL - ded
2020-09-06 12:06:59,742 | Bad name | CRITICAL - not big surprise