CODEWITHSUNDEEP
pythonfileloggingstream
frei
frei

Reputation: 536

logging with dictConfig and writing to the console and a file

In my code I have the following for a verbose mode and a non-verbose mode. I'm reading from a logDict object.

I expect that in verbose mode I will get "DEBUG MODE: test debug" and "DEBUG MODE: test error" written to the console and "[uuid] [date] [etc] test error" only written to a file, and that in non-verbose mode that nothing gets printed to the console but "test error" will be written to the file.

First, here is my dictConfig

LOGGING_DICT = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
    'simple': {
        # we have a uuid for the log so we know what process it came from
        'format': '[{0}][%(asctime)s][%(name)s][%(levelname)s] : %(message)s'.format(logger_id),
        'datefmt': "%Y-%m-%d %H:%M:%S",
        }
},
'loggers': {
    'root': {
        'handlers': ['console'],
        'level': 'DEBUG',
    },
    'script_A': {
        'handlers': ['timed_rotate_file'],
        'level': 'INFO',
    },
},
'handlers' : {
    'timed_rotate_file': {
        'filename': 'logs/weekly_tool.log',
        'level': 'INFO',
        'formatter': 'simple',
        'class': 'logging.handlers.TimedRotatingFileHandler',
        'encoding': 'utf8',
        # Used to configure when backups happen 'seconds, minutes, w0,w1 (monday tuesday)
        'when': 'midnight',  # Daily backup
        # This is used to configure rollover (7=weekly files if when = daily or midnight)
        'backupCount': 7,
    }
}

And now the script that calls it

from logging.config import dictConfig
from helpers.logging_config import LOGGING_DICT

...
main():
    logger.debug("test debug")
    logger.error("test error")

if __name__ == "__main__":
    if args.verbose:
        dictConfig(LOGGING_DICT)
        logger = logging.getLogger("script_A")
        stream_handler = logging.StreamHandler()
        formatter = logging.Formatter("DEBUG MODE: %(message)s")
        stream_handler.setFormatter(formatter)
        stream_handler.setLevel(logging.DEBUG)
        logger.addHandler(stream_handler)
    else:
        dictConfig(LOGGING_DICT)
        logger = logging.getLogger("script_A")

What I get instead is the following:

~$ python script_A.py
~$ (No output, as expected)
~$ python script_A.py -v
~$ DEBUG MODE: test error

Why is the test_debug not printing to console? Clearly the stream handler is being called, but the level is either not being set correctly or is being ignored.

When I print logger.level in the middle of the script I get 20, which is what I expect given dictConfig, however the handler's level is being set separately, does that mean it is being ignored? (What is the point of setLevel in a python logging handler?) <-- I'm looking at this as well, but my issue is flipped. In dict config my settings are stricter than what I actually want to print, which means that if I reset my log level for the logger I'm getting from dictConfig, things I don't want to print to my file are going to be printed. Can I circumvent this?

Upvotes: 2

Views: 2125

Answers (1)

frei
frei

Reputation: 536

I figured this out on my own. Similar to what I posted, I have to reset the log level.

if __name__ == "__main__":
    if args.verbose:
        dictConfig(LOGGING_DICT)
        logger = logging.getLogger("script_A")
        stream_handler = logging.StreamHandler()
        formatter = logging.Formatter("DEBUG MODE: %(message)s")
        stream_handler.setFormatter(formatter)
        stream_handler.setLevel(logging.DEBUG)
        logger.addHandler(stream_handler)
        logger.setLevel(logging.DEBUG)
    else:
        dictConfig(LOGGING_DICT)
        logger = logging.getLogger("script_A")

I thought that doing this would mean that the file handler level also gets changed, but for some reason that doesn't happen. If anyone knows why I would love to know how the internals work this out.

Upvotes: 2

Related Questions