CODEWITHSUNDEEP
python
robertlayton
robertlayton

Reputation: 722

logging: print message only once

I'm using the logging module in python 3, and want a particular warning message to be displayed only once. The problem is that the check is inside a loop:

def canned_example():
    logger.warning("This function hasn't been created yet")


for i in range(10):
    canned_example()

Is there a flag to set inside the logging module that will denote that this particular warning is only to be displayed once? The alternative is to keep a record of different flags, but I'd like to keep it simple if possible.

Update: Amir Yazdanbakhsh has posted an answer in the comments that allows us to do this for all messages. Ideally, I'd like some per-message flag:

def canned_example():
    logger.warning("This function hasn't been created yet", norepeat=True)


for i in range(10):
    canned_example()

Upvotes: 30

Views: 14355

Answers (4)

jakob-r
jakob-r

Reputation: 7282

Another solution that uses attributes you can attach to an object (ultimately a function is also an object in python)

def logging_warning_once(message):
    if not hasattr(logging_warning_once, "logged"):
        logging_warning_once.logged = set()
    if message not in logging_warning_once.logged:
        logging.warning(message)
        logging_warning_once.logged.add(message)

Upvotes: 1

Kound
Kound

Reputation: 2551

A bit of a hack but much easier is "misusing" functools lru_cache.

from functools import lru_cache
from logging import getLogger

# Keep track of 10 different messages and then warn again
@lru_cache(10)
def warn_once(logger: Logger, msg: str):
    logger.warning(msg)

You can increase the 10 to suppress more if required or set it to None to store suppress everything duplicated.

This answer also full fill the requirement to only suppress only specific duplicate warning, as logger.warning("message") is not filtered at all.

Also this solution is more memory efficient as duplicated logs don't remain in memory multiple times.

Upvotes: 32

erb
erb

Reputation: 15551

This is a modified variant of @chepner's answer. I had some issues getting it to work when the thing logged wasn't a string (i.e. an exception) and wanted to make use of context managers to more conveniently add and remove the filter:

class DuplicateFilter:
    """
    Filters away duplicate log messages.
    Modified version of: https://stackoverflow.com/a/31953563/965332
    """

    def __init__(self, logger):
        self.msgs = set()
        self.logger = logger

    def filter(self, record):
        msg = str(record.msg)
        is_duplicate = msg in self.msgs
        if not is_duplicate:
            self.msgs.add(msg)
        return not is_duplicate

    def __enter__(self):
        self.logger.addFilter(self)

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.logger.removeFilter(self)

Usage:

logger = logging.getLogger(__name__)

with DuplicateFilter(logger):
    for _ in range(5):
        logger.info("This will only show up once!")
        logger.error(Exception("So will this!"))   # This didn't work in @chepner's example

Upvotes: 6

chepner
chepner

Reputation: 531993

Define a filter which keeps track of what was logged, and attach it to your logger for the duration of the loop. This example will remember each message it sees, and only allow the first occurrence to be logged.

class DuplicateFilter(object):
    def __init__(self):
        self.msgs = set()

    def filter(self, record):
        rv = record.msg not in self.msgs
        self.msgs.add(record.msg)
        return rv

dup_filter = DuplicateFilter()
logger.addFilter(dup_filter)
for i in range(10):
    canned_example()
logger.removeFilter(dup_filter)

Upvotes: 19

Related Questions