CODEWITHSUNDEEP
pythonlogging
Shanpei Zhou
Shanpei Zhou

Reputation: 391

python logging to multiple destination

I am new to Python. I now need to make use of logging and here is the experiment I did:

I have 2 files:

logtest_1.py, logtest_2.py

logtest_1.py:

    import logging
    from logtest_2 import loglib

    format = '%(asctime)s: %(levelname)s: %(message)s'
    logging.basicConfig(format=format,level=logging.DEBUG)
    logger = logging.getLogger(__name__)
    logger.info('Before calling loglib...')
    loglib()
    logger.info('Loglib done.')

logtest_2.py: import logging from logging.handlers import TimedRotatingFileHandler import time

def loglib():
    logger_2 = logging.getLogger(__name__)

    logger_3 = logging.getLogger('TimeRotateLogger')
    logger_3.setLevel(logging.DEBUG)
    handler = TimedRotatingFileHandler('timerotate.log',
                                        when='s',
                                        interval=2,
                                        backupCount=10)
    logger_3.addHandler(handler)

    logger_2.info('This is a log from logtest_2.')

    time.sleep(1)
    for i in range(5):
        logger_3.info('Rotation test...')
        time.sleep(2)

I try to use logger_3 to write info to those time rotation files. Actually, this works. However, it also prints out to the screen:

2015-02-16 15:26:34,010: INFO: Before calling loglib...
2015-02-16 15:26:34,011: INFO: This is a log from logtest_2.
2015-02-16 15:26:35,019: INFO: Rotation test...
2015-02-16 15:26:37,029: INFO: Rotation test...
2015-02-16 15:26:39,039: INFO: Rotation test...
2015-02-16 15:26:41,049: INFO: Rotation test...
2015-02-16 15:26:43,059: INFO: Rotation test...
2015-02-16 15:26:45,070: INFO: Loglib done.

I only want logger_3 to log into those files. How can I prevent it from printing to the screen?

Also, why this is happening? I've already given the handler to logger_3 which writes to files.

If I really want to keep logging.basicConfig(format=format,level=logging.DEBUG) in the logtest_1.py, what should I do?

Upvotes: 1

Views: 2682

Answers (2)

Lukas Graf
Lukas Graf

Reputation: 32690

This happens because your logger_3 by default also propagates log events to its parent logger, the root logger.

If you use basicConfig(), the root logger will have a StreamHandler attached to it by default which causes your message to also end up on the console.

To prevent this, you can either set logger_3.propagate = False, or only attach handlers directly to your root logger (which is the most common used setup) and use logging levels and filters to control where your output goes.

An example for only attaching your handlers to the root logger could look like this:

import logging
from logging.handlers import TimedRotatingFileHandler


format = '%(asctime)s: %(levelname)s: %(message)s'
logging.basicConfig(format=format, level=logging.INFO)
logger = logging.getLogger(__name__)

handler = TimedRotatingFileHandler('timerotate.log',
                                   when='s',
                                   interval=2,
                                   backupCount=10)
handler.setLevel(logging.DEBUG)
logging.root.addHandler(handler)

logger.info('This will end up on console and in timerotate.log')
logger.debug('This will only end up in timerotate.log')

basicInfo() attaches a StreamHandler with level INFO to the root logger that logs to stdout - but only messages with level INFO or higher.

You then attach a second handler with level DEBUG, so it will log every message with level DEBUG or higher to timerotate.log. This is a very simple example that uses different log levels to determine where output ends up - using this approach you can't send a particular statement only to the console, and a different one only to a file.

If you need more fine grained control, you could for example pass some data in the extra keyword argument when logging, and add a filter to any of your handlers to only emit messages that match certain criteria.

But, if you simply want to be able to directly control from your code that a particular message should go to a file, and only that file, then it's probably easiest to simply set logger_3.propagate to False, and use the setup you already described.

Upvotes: 3

Iguananaut
Iguananaut

Reputation: 23376

See Logger.propagate. Set this to False on your submodule loggers to prevent messages logged to them from bubbling up to your root logger as well.

Upvotes: 1

Related Questions