CODEWITHSUNDEEP
pythoninheritancelogging
proximous
proximous

Reputation: 677

Python Logging with a common logger class mixin and class inheritance

I would like to create a Python logging class that can be inherited as a common means of logging configuration, yet seperately control the logging level of the base class from the parent. This is similar to How to use python logging in multiple modules. The answer by Vinay Sajip to use a LogMixin is very close. Below is my slightly modified version.

Most of my classes inherit smaller classes. For example:

filename: LogMixin.py

import logging, logging.config
import yaml
class LogMixin(object):
    __loggerConfigured = False
    @property
    def logger(self):
        if not self.__loggerConfigured:
            with open('log_config.yaml', 'rt') as f:
                config = yaml.load(f.read())
                logging.config.dictConfig(config)
            self.__loggerConfigured = True
        name = '.'.join([self.__class__.__name__])
        return logging.getLogger(name)

filename: Base.py

from LogMixin import LogMixin
class Base(LogMixin):
    def __init__(self):
        self.logger.debug("Debug Base")
    def run_base(self):
        self.logger.debug("Debug Running Base")
        self.logger.info("Info Running Base")
if __name__ == '__main__':
    my_base = Base()
    my_base.run_base()

filename: Parent.py

from Base import Base
class Parent(Base):
    def __init__(self):
        self.logger.debug("Debug Parent")
    def run_parent(self):
        self.logger.debug("Debug Running Parent")
        self.logger.info("Info Running Parent")

if __name__ == '__main__':
    my_parent = Parent()
    my_parent.run_base()
    my_parent.run_parent()

filename: log_config.yaml

---
version: 1
disable_existing_loggers: False

# Configuring the default (root) logger is highly recommended
root:
    level: WARNING
    handlers: [console]

# Configuration for logger set with logging.getLogger(NAME)
loggers:
    Base:
        level: INFO
        handlers: [console]
        propagate: no
    Parent:
        level: DEBUG
        handlers: [console]
        propagate: no

formatters:
    simple:
        format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"

handlers:
    console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
...

I get the benefits of the common logging configuration. However, I'd like independent control of the log levels for both Base and Parent. With the config file above, I get:

$ python Base.py                 
2015-03-16 00:06:23,716 - Base - INFO - Info Running Base
$ python Parent.py                
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Parent
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Running Base
2015-03-16 00:06:19,682 - Parent - INFO - Info Running Base
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Running Parent
2015-03-16 00:06:19,682 - Parent - INFO - Info Running Parent

I understand why I get this, I only have one logger "Parent". However, in general, I'd rather get the following:

$ python Base.py                 
2015-03-16 00:06:23,716 - Base - INFO - Info Running Base
$ python Parent.py                
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Parent
2015-03-16 00:06:19,682 - Base - INFO - Info Running Base
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Running Parent
2015-03-16 00:06:19,682 - Parent - INFO - Info Running Parent

(notice no DEBUG related to Base.py).
Or even better:

$ python Base.py                 
2015-03-16 00:06:23,716 - Base - INFO - Info Running Base
$ python Parent.py                
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Parent
2015-03-16 00:06:19,682 - Parent.Base - INFO - Info Running Base
2015-03-16 00:06:19,682 - Parent - DEBUG - Debug Running Parent
2015-03-16 00:06:19,682 - Parent - INFO - Info Running Parent

(Notice the name is Parent.Base so I can see the inheritance.) Is this possible with a single simple LogMixin class?

Upvotes: 13

Views: 11258

Answers (1)

Guy Gangemi
Guy Gangemi

Reputation: 1773

In your example, when self.logger is invoked, it makes a call to self.__class__.__name__. That's always going to be the first class in the mro, i.e the class that was called to create the instance.

The solution to this is name mangling which ensures that a classes method uses that same classes attribute.

import logging

class Base:
    __logger = logging.getLogger('Base')

    def __init__(self):
        self.__logger.error('init base')

    def func_base(self):
        self.__logger.error('func base')

class Parent(Base):
    __logger = logging.getLogger('Base.Parent')

    def func_parent(self):
        self.__logger.error('func parent')

A metaclass can be used to ensure the logger is created and attributed to the class, once, when it is defined.

import logging
import sys

logging.basicConfig(stream=sys.stdout)

class MetaBase(type):
    def __init__(cls, *args):
        super().__init__(*args)

        # Explicit name mangling
        logger_attribute_name = '_' + cls.__name__ + '__logger'

        # Logger name derived accounting for inheritance for the bonus marks
        logger_name = '.'.join([c.__name__ for c in cls.mro()[-2::-1]])

        setattr(cls, logger_attribute_name, logging.getLogger(logger_name))

class Base(metaclass=MetaBase):
    def __init__(self):
        self.__logger.error('init base')

    def func_base(self):
        self.__logger.error('func base')

class Parent(Base):
    def func_parent(self):
        self.__logger.error('func parent')

p = Parent()
p.func_base()
p.func_parent()

Results in:

ERROR:Base:init base
ERROR:Base:func base
ERROR:Base.Parent:func parent

I've simplified the example to demonstrate the key concept. Should work across files and with a config file.

Upvotes: 8

Related Questions