CODEWITHSUNDEEP
pythonlogging
cjm2671
cjm2671

Reputation: 19476

Logging settings overwritten by Python module

I'm a little bit confused about how best to approach setting up logging in my Python application.

I'm using the IBPy module, which contains it's own logging here:

import logging
import os  

format = '%(asctime)s %(levelname)-9.9s %(message)s'

datefmt = '%d-%b-%y %H:%M:%S'

##
# Default log level.  Set IBPY_LOGLEVEL environment variable to
# change this default.
level = int(os.environ.get('IBPY_LOGLEVEL', logging.DEBUG))


def logger(name='ibpy', level=level, format=format,
               datefmt=datefmt):
    logging.basicConfig(level=level, format=format, datefmt=datefmt)
    return logging.getLogger(name)

My application consists of a bunch of files that are imported into a Jupyter notebook. Final execution takes place inside the notebook.

Currently, other modules are firing off on the 'DEBUG' level, and my notebook is full of warnings.

What is the right approach to configure logging at the module level?

Upvotes: 4

Views: 1556

Answers (2)

Guy Gangemi
Guy Gangemi

Reputation: 1773

IbPy's call to basicconfig has configured 'root' logger to catch events with debug level and above and send them to stderr. That effectively results in every event from every logger being processed. There's a few ways to get this back under control.

Mix and match the following at the top of your own module.

Set ibpy level

Is ibpy polluting your console? Set just the ibpy logger to level 'warning'. You will still get debug or greater events for all other loggers.

logging.getLogger('ibpy').setLevel('WARNING')

Set root level

Restrict the root logger to only process info and above. This effectively means you'll see info and above events from all loggers.

logging.getLogger().setLevel('INFO')

Remove root handler

Remove the handler from root, nothing goes to stderr.

logging.getLogger().handlers.clear()

Add own logger

May need to configure a new handler if you removed the one from root. Consider adding it to this logger if you want to avoid logs from any other logger.

logger = logging.getLogger('mine')

Upvotes: 2

Drachenfels
Drachenfels

Reputation: 3286

Solutions provided by @Guy Gangemi should be working from my quick look at them. That being said there could be a simple race condition, before you have a chance to update logging, IBPY is already spamming in your console.

I would argue that this is the problem with the library itself, it should not configure root logger, all they wanted to do is to add formatter and log level and they could and should do it to their own logger ibpy.

That being said overriding env variable should work with a caveat, you need to make sure that IBPY_LOGLEVEL is set before anything or anyone imports logger.py once this import is executed log_level is set in stone and it won't change.

The final solution is to wrap their code with one of yours if you control where and when you import their package you can force root logging configuration in advance:

    log_config = {
        "version": 1,
        "disable_existing_loggers": False,  # you can change that to True for a good measure
        "formatters": {
            "generic": {
                "format": "%(levelname)-5.5s [%(name)s] %(message)s",
                "datefmt": "%H:%M:%S",
            },
        },
        "handlers": {
            "dev_console": {
                "class": "logging.StreamHandler",
                "formatter": "generic",
            },
        },
        "loggers": {
            "": {
                "handlers": ['dev_comsole'],
                "level": logging.WARN,
                "propagate": True,
            },
        },
    }
    logging.config.dictConfig(log_config)

All in all package seems to be obsolete, not the best written and abandoned, so perhaps there are better tools that can achieve the same, minus headache.

Upvotes: 0

Related Questions