How to only log a certain levelname python logging

Question

[loggers]
keys=root

[handlers]
keys=consoleHandler,errorHandler,debugHandler

[formatters]
keys=errorFormatter,debugFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler,errorHandler,debugHandler
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=debugFormatter
args=(sys.stdout,)

[handler_errorHandler]
class=FileHandler
level=ERROR
formatter=errorFormatter
args=('errors.log',)

[handler_debugHandler]
class=FileHandler
level=DEBUG
formatter=debugFormatter
args=('debugs.log',)

[formatter_errorFormatter]
style={
format=
    {asctime} - {name} - {levelname} - line {lineno} - {message}
datefmt=%d/%m/%y - %H:%M:%S

[formatter_debugFormatter]
style={
format={asctime} - {name} - {levelname} - {message}
datefmt=%d/%m/%y - %H:%M:%S

If I have a config file like above, how can I access the debugHandler so that I can do like debugHandler.addFilter(my_custom_filter)?

I'm essentially trying to make it so that DEBUG messages are sent to debugs.log and ERROR messages are sent to errors.log, but currently ERROR messages are also sent to debugs.log which I don't want. Now, based on this StackOverflow post, this is achievable via filters but in order to add a filter I need a logging.Handler object and I don't know how to obtain one.

Thanks in advance :~)

Answer 1

Yep, you would need a filter on your debug_handler, however you cannot add Filters via a fileConfig.

From the docs:

The fileConfig() API is older than the dictConfig() API and does not provide functionality to cover certain aspects of logging. For example, you cannot configure Filter objects, which provide for filtering of messages beyond simple integer levels, using fileConfig(). If you need to have instances of Filter in your logging configuration, you will need to use dictConfig(). Note that future enhancements to configuration functionality will be added to dictConfig(), so it’s worth considering transitioning to this newer API when it’s convenient to do so.

So consider using dictConfig (there shouldn't be any reason for you not to switch to dictConfig, regardless of your configuration, as it is better, newer, more flexible, and the one where all the future good stuff will be added).

So your config, transitioned to dictConfig would look something like this (this can be pasted and tested as-is):

import logging
from logging import config

class FileFilter:
    """Allow only LogRecords whose severity levels are below ERROR."""

    def __call__(self, log):
        if log.levelno < logging.ERROR:
            return 1
        else:
            return 0


logging_config = {
    'version': 1,
    'formatters': {
        'error_formatter': {
            'format': '{asctime} - {name} - {levelname} - line {lineno} - {message}',
            'style': '{',
            'datefmt': '%d/%m/%y - %H:%M:%S',
        },
        'debug_formatter': {
            'format': '{asctime} - {name} - {levelname} - {message}',
            'style': '{',
            'datefmt': '%d/%m/%y - %H:%M:%S',
        },
    },
    'filters': {
        'file_filter': {
            '()': FileFilter,
        },
    },
    'handlers': {
        'console_handler': {
            'class': 'logging.StreamHandler',
            'formatter': 'debug_formatter',
        },
        'error_handler': {
            'class': 'logging.FileHandler',
            'formatter': 'debug_formatter',
            'level': 'ERROR',
            'filename': 'errors.log',
        },
        'debug_handler': {
            'class': 'logging.FileHandler',
            'formatter': 'debug_formatter',
            'filters': ['file_filter'],
            'filename': 'debug.log',
        },
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['console_handler', 'error_handler', 'debug_handler'],
    },
}

config.dictConfig(logging_config)

logger = logging.getLogger(__name__)

# these get logged to the console and only to the debugs.log file
# if you want just the debug messages logged to the file, adjust the filter
logger.debug('this is a debug message')
logger.info('this is an info message')
logger.warning('this is a warning message')

# this get logged to the console and only to the errors.log file
logger.error('this is an error message')
logger.critical('this is a critical message')

Which will output:

27/09/19 - 07:52:44 - __main__ - DEBUG - this is a debug message  # also to debug.log
27/09/19 - 07:52:44 - __main__ - INFO - this is an info message  # also to debug.log
27/09/19 - 07:52:44 - __main__ - WARNING - this is a warning message  # also to debug.log
27/09/19 - 07:52:44 - __main__ - ERROR - this is an error message  # also to errors.log but not to debug.log
27/09/19 - 07:52:44 - __main__ - CRITICAL - this is a critical message  # also to errors.log but not to debug.log

If you, for some mysterious reason (I can't see any) insist on using the older API, fileConfig, see this answer which uses custom formatters, which achieve the same thing, without filters.