CODEWITHSUNDEEP
pythonloggingsingleton
Hussain
Hussain

Reputation: 5187

Python singleton logging not working properly - Duplicate log statements in the file

I am trying to put logging in python python project. For that, I have written a method called get_logger in my common.py that returns the logger.

import logging
def get_logger(name='app'):
    log_config = config.LOG_CONFIG
    logging.config.dictConfig(log_config)
    logger = logging.getLogger(name)
    return logger

Note that the log configurations are put in config.LOG_CONFIG 

LOG_CONFIG = {
    'version': 1,
    'disable_existing_loggers': False,  # this fixes the problem

    'formatters': {
        'standard': {
            'format': '%(asctime)s [%(levelname)8s] %(name)s: %(message)s'
        },
        # more formatters
    },
    'handlers': {
        'file': {
            'filename': '%s/%s' % (_cwd, '../logs/app/logging.log'),
            'formatter': 'standard',
            'when': 'midnight',
            'class': 'logging.handlers.TimedRotatingFileHandler'
        },
    },
    'loggers': {
        # Works for
        # 1. application logs (flask) and
        # 2. all other logs where below logger name is passed to getLogger
        'app': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True
        },
        # For all the remaining logs
        # Important: Don't include for production??
        '': {
            'handlers': ['file'],
            'level': 'INFO',
            'propagate': True
        }
    }
}

I have two modules

a class from sf_apis.py extends a class from base.py. I have called get_logger in base.py.

base.py

from common import get_logger
log = get_logger()

class Base:

    // .. some code ..

Then just called getLogger from logging to get that logger in sf_apis.py

sf_apis.py

import logging

log = logging.getLogger('app')

class SfBulk(Base):

    // .. some code ..

But when my another module called sync_data.py that runs by calling some methods from sf_apis.py, I see duplicate log statements in my log

2015-11-04 13:05:58,605 [    INFO] app: close_the_job :: job id = 75090000003sf4YAAQ
2015-11-04 13:05:58,605 [    INFO] app: close_the_job :: job id = 75090000003sf4YAAQ

I read it here that

The logging module already implements a singleton pattern

so I didn't go and create a singleton Logger class (There are many examples on internet)

What did I miss? Please help.

Upvotes: 2

Views: 1203

Answers (1)

Anton Zuenko
Anton Zuenko

Reputation: 761

Python loggers have hierarchical structure. You've created two: 'app' and ''. By convention, hierarchy is defined as dot-separated path to the module. In a particular case, empty string logger becomes a parent for all other first-level loggers. Specifically, '' is parent of 'app'.

Now you set propagate = True for 'app' logger. It means that a message sent to this logger would be also propagated to its parent, i.e. ''. That's how you have duplicated messages.

You can either set propagate = False to 'app' logger or set another file handler for '' logger such that the messages would be still duplicated, but in different files.

Upvotes: 2

Related Questions