extending logging.Logger module in python 3.5

Question

I have been trying to create a new class of Logger by subclassing logging.Logger . Python version is 3.5

I have several modules in my application and I configure the logging only in the main module where I set the logger class using logging.setLoggerClass(...)

However when I retrieve the same Logger instance from some other module, it still creates a new instance of the Logger class and not the child class instance that I defined.

For example my code is :

# module 1
import logging
class MyLoggerClass(logging.getLoggerClass()):
     def __init__(name):
         super(MyLoggerClass, self).__init__(name)

     def new_logger_method(...):
         # some new functionality

if __name__ == "__main__":
    logging.setLoggerClass(MyLoggerClass)
    mylogger = logging.getLogger("mylogger")
    # configuration of mylogger instance

# module 2
import logging
applogger = logging.getLogger("mylogger")
print(type(applogger))
def some_function():
     applogger.debug("in module 2 some_function")

When this code is executed, I expect the applogger in module 2 to be of type MyLoggerClass. I intend to use the new_logger_method for some new functionality.

However since applogger is turning out to be of type logging.Logger, when the code is run it throws Logger has no attribute named new_logger_method.

Has anyone ever faced this issue?

Thanks in advance for any help! Pranav

Answer 1

Instead of attempting to affect the global logger by changing the default logger factory, if you want your module to play nicely with any environment you should define a logger just for your module (and its children) and use it as a main logger for everything else deeper in your module structure. The trouble is that you explicitly want to use a different logging.Logger class than the default/globally defined one and the logging module doesn't provide an easy way to do context-based factory switching so you'll have to do it yourself.

There are many ways to do that but my personal preference is to be as explicit as possible and define your own logger module which you'll then import in your other modules in your package whenever you need to obtain a custom logger. In your case, you can create logger.py at the root of your package and do something like:

import logging

class CustomLogger(logging.Logger):

    def __init__(self, name):
        super(CustomLogger, self).__init__(name)

    def new_logger_method(self, caller=None):
        self.info("new_logger_method() called from: {}.".format(caller))

def getLogger(name=None, custom_logger=True):
    if not custom_logger:
        return logging.getLogger(name)
    logging_class = logging.getLoggerClass()  # store the current logger factory for later
    logging._acquireLock()  # use the global logging lock for thread safety
    try:
        logging.setLoggerClass(CustomLogger)  # temporarily change the logger factory
        logger = logging.getLogger(name)
        logging.setLoggerClass(logging_class)  # be nice, revert the logger factory change
        return logger
    finally:
        logging._releaseLock()

Feel free to include other custom log initialization logic in it if you so desire. Then from your other modules (and sub-packages) you can import this logger and use its getLogger() to obtain a local, custom logger. For example, all you need in module1.py is:

from . import logger  # or `from package import logger` for external/non-relative use

log = logger.getLogger(__name__)  # obtain a main logger for this module

def test():  # lets define a function we can later call for testing
    log.new_logger_method("Module 1")

This covers the internal use - as long as you stick to this pattern in all your modules/sub-modules you'll have the access to your custom logger.

When it comes to external use, you can write an easy test to show you that your custom logger gets created and that it doesn't interfere with the rest of the logging system therefore your package/module can be declared a good citizen. Under the assumption that your module1.py is in a package called package and you want to test it as a whole from the outside:

import logging  # NOTE: we're importing the global, standard `logging` module
import package.module1

logging.basicConfig()  # initialize the most rudimentary root logger
root_logger = logging.getLogger()  # obtain the root logger
root_logger.setLevel(logging.DEBUG)  # set root log level to DEBUG

# lets see the difference in Logger types:
print(root_logger.__class__)  # <class 'logging.RootLogger'>
print(package.module1.log.__class__)  # <class 'package.logger.CustomLogger'>

# you can also obtain the logger by name to make sure it's in the hierarchy
# NOTE: we'll be getting it from the standard logging module so outsiders need
#       not to know that we manage our logging internally
print(logging.getLogger("package.module1").__class__) # <class 'package.logger.CustomLogger'>

# and we can test that it indeed has the custom method:
logging.getLogger("package.module1").new_logger_method("root!")
# INFO:package.module1:new_logger_method() called from: root!.
package.module1.test()  # lets call the test method within the module
# INFO:package.module1:new_logger_method() called from: Module 1.

# however, this will not affect anything outside of your package/module, e.g.:
test_logger = logging.getLogger("test_logger")
print(test_logger.__class__)  # <class 'logging.Logger'>
test_logger.info("I am a test logger!")
# INFO:test_logger:I am a test logger!
test_logger.new_logger_method("root - test")
# AttributeError: 'Logger' object has no attribute 'new_logger_method'