CODEWITHSUNDEEP
pythonloggingconfig
Mike Williamson
Mike Williamson

Reputation: 3198

Python `logging` module + YAML config file

I have found a lot of documentation and tutorials such as the official logging config docs, the official logging cookbook, and this nice tutorial by Fang.

Each of them have gotten me near to an answer, but not quite. My question is this:

When using Config Files, how can I use a logger with 2 separate handlers at 2 separate levels?

To clarify, here is an example of my YAML file:

---
version: 1
handlers:
  debug_console:
    class: logging.StreamHandler
    level: DEBUG
.
.
.

  info_file_handler:
    class: logging.handlers.RotatingFileHandler
    level: INFO
.
.
.

loggers:
  dev:
    handlers: [debug_console, info_file_handler]
  test:
    handlers: [info_file_handler]
root:
  handlers: [info_file_handler]

I want to have two ways to run the logger, where one way (dev) is more verbose than the other. Moreover, when running the dev logger, I want it to have two different levels for the two different handlers.

This is a snippet of the code to try to launch the logger:

with open('logging.yaml', 'r') as f:
  log_cfg = yaml.safe_load(f.read())
  logging.config.dictConfig(log_cfg)
  my_logger = logging.getLogger('dev')

The dictConfig line above works correctly. I say this because when I get to the code which asks to log to the console, I will see dev as the name when the log prints out. (I have edited the yaml, but it contains %(name)s in the format.)

But there is something wrong with my_logger. Even though it is tied to the name of dev, none of the rest of the attributes seem to have been set. Specifically, I see:

>>> my_logger
<Logger dev_model (WARNING)>

I don't know the logging module well enough to understand where the problem is. What I want is:

When I activate the 'dev' logger, I want to launch 2 handlers, one which is at the DEBUG level and writes to console, the other which is at the INFO level and writes to a file.

How can this be done?

Upvotes: 0

Views: 5051

Answers (2)

dvk
dvk

Reputation: 1480

If I understand the question correctly, the problem is caused by the fact the logger itself has a log level, not just handlers. Logger's log level defaults to WARNING, which seems to be set on your logger. If a generated message has a lower priority than the logger's level then it does not even make to the handlers.

So try setting logger's level to DEBUG. info_file_handler should ignore any messages more verbose than it's own level.

As for this part:

none of the rest of the attributes seem to have been set.

What happens there is logger's repr() method is called to convert Logger to some sort of string representation in order to render it. Which is not guaranteed to show all the attributes of the object.

Upvotes: 2

Sraw
Sraw

Reputation: 20214

Such a long question... Too long for me to understand well.

But I think you misunderstand how handlers work. Actually logger itself doesn't output anything but handlers do.

So let's say if you set DEBUG on dev logger, it will pass logs >= DEBUG to all handlers. And then debug_console handler will process logs >= DEBUG but info_file_handler will only process logs >= INFO. Setting DEBUG on dev logger won't let info_file_handler output logs < INFO. So you do can have two separate levels which one is >= DEBUG and goes to console while another is >= INFO and goes to file.

I am presuming I understand you rightly...

Upvotes: 0

Related Questions