In Python, how to I get an extended class element to show up in autocomplete?

Question

I'm new to classes so using an example I found online to add some custom logging levels. This will be contained in a library which will be importrd into various scripts. It is working as expected but the added levels don't show up in the autocomplete list (using PyCharm) and PyCharm complains of an unresolved attribute reference in LOGGER. When I'm coding and enter 'LOGGER.' I see the normal error, warning, info, etc. to choose from but my custom level 'verbose' is not in the list. There will be more custom levels added as time go by and this will also be rolled out to a team of developers so I need to have this working.

Any idea why verbose is not an option in my autocomplete list?

Here are my files.

px_logger.py

from logging import getLoggerClass, addLevelName, setLoggerClass, NOTSET

public class PxLogger(getLoggerClass()):
    def __init__(self, name, level=NOTSET):
        super(PxLogger, self).__init__(name, level)

        addLevelName(5, "VERBOSE")

    def verbose(self, msg, *args, **kwargs):
        """Custom logger level - verbose"""
        if self.isEnabledFor(5):
            self._log(5, msg, args, **kwargs)

my_script.py

import json
import logging.config
from px_logger import PxLogger

logging.setLoggerClass(PxLogger)
LOGGER = logging.getLogger(__name__)

with open('../logging.json') as f:  # load logging config file
    CONFIG_DICT = json.load(f)
logging.config.dictConfig(CONFIG_DICT)

LOGGER.verbose('Test verbose message')

Screen output

VERBOSE - Test verbose message

Answer 1

PyCharm offers various ways to accomplish type hinting

Internally, it uses Typeshed to determine the types of standard lib objects and common 3rd party packages. That's also where PyCharm takes the type of the return value for logging.getLogger from and that's why it does not show your subclass' verbose method in autocomplete, because it assumes LOGGER to be an instance of logging.Logger.

The easiest way to tell PyCharm's type checker that LOGGER is an instance of PxLogger would be a type annotation in the code during assignment. This works in Python 3.5+ only:

LOGGER: PxLogger = logging.getLogger(__name__)

If you went one step further, you would encapsulate the definition of your custom logger class, it being assigned as global logger class and the definition of a wrapper for logging.getLogger inside your module.

This would enable your coworkers to just import your module instead of logging and use it just as they would with the original logging without having to worry about which class to set as logger class or how to annotate the variable that holds their logger.

There's three options to include type hinting for the type checker when going down this road.

px_logger.py

# basically, import from logging whatever you may need and overwrite where necessary
from logging import getLogger as _getLogger, Logger, addLevelName, setLoggerClass, NOTSET
from typing import Optional # this only for the Python 3.5+ solution

class PxLogger(Logger):  # Note: subclass logging.Logger explicitly
    def __init__(self, name, level=NOTSET):
        super(PxLogger, self).__init__(name, level)

        addLevelName(5, "VERBOSE")

    def verbose(self, msg, *args, **kwargs):
        """Custom logger level - verbose"""
        if self.isEnabledFor(5):
            self._log(5, msg, args, **kwargs)

setLoggerClass(PxLogger)

"""
Possible approaches, implement one of the below.
The first is Python 3.5+ only.
The second and third work for both, Python 2 and Python 3.
"""
# using type annotation syntax (py35+)
def getLogger(name: Optional[str]=None) -> PxLogger:
    _logr: PxLogger = _getLogger(name)
    return _logr

# using (legacy) docstring syntax (py2and3)
def getLogger(name=None)
    """
    :param name: str
    :rtype: PxLogger
    """ 
    return _getLogger(name)

# using a stub file (py2and3)
def getLogger(name=None):
    return _getLogger(name)

The Python 2and3 stub file approach requires a file named py_logger.pyi next to the actual module file px_logger.py in your package.

px_logger.pyi

# The PEP-484 syntax does not matter here. 
# The Python interpreter will ignore this file, 
# it is only relevant for the static type checker
import logging

class PxLogger(logging.Logger):
    def verbose(self, msg, *args, **kwargs) -> None: ...

def getLogger(name) -> PxLogger: ...

For all three approaches, your module my_script would look the same:

my_script.py

import logging.config
import px_logger

LOGGER = px_logger.getLogger(__name__)

# I used basicConfig here for simplicity, dictConfig should work just as well
logging.basicConfig(level=5,
                    format='%(asctime)s - %(levelname)s [%(filename)s]: %(name)s %(funcName)20s - Message: %(message)s',
                    datefmt='%d.%m.%Y %H:%M:%S',
                    filename='myapp.log',
                    filemode='a')

LOGGER.verbose('Test verbose message')

Autocomplete works well with all three approaches:

Approach two and three have been tested with Python 2.7.15 and 3.6.5 in PyCharm 2018.1 CE

NOTE: In a previous revision of this answer I stated that the docstring approach, although showing the custom verbose() method of your PxLogger class, is missing the base class' methods. That is because you derive PxLogger from whatever logging.getLoggerClass returns, i.e. an arbitrary class. If you make PxLogger a subclass of logging.Logger explicitly, the type checker knows the base class and can correctly resolve its methods for autocompletion.

I do not recommend subclassing the return value of getLoggerClass anyway. You'll want to be sure what you derive from and not rely on a function returning the correct thing.

Answer 2

Annotate the variable with a typing.

LOGGER: PxLogger = logging.getLogger(__name__)

EDIT:

The solution above works for Python 3.6+. Use # type: type comments for previous versions, as noted by @pLOPeGG in comments.

LOGGER = logging.getLogger(__name__)  # type: PxLogger