CODEWITHSUNDEEP
pythonloggingyaml
Dave
Dave

Reputation: 454

Displaying requests module logging messages whilst using a .yaml config file for logging module

How might I display the requests module logging messages whilst using a .yaml file to configure the logging module?

Previously I have used the following to display logging output from requests, but I'm now lost:

logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s - %(levelname)s - %(message)s")
#logging.disable(logging.CRITICAL)

My .yaml file, which has replaced the above code:

version: 1
disable_existing_loggers: False
handlers:
  console:
    class: logging.StreamHandler
    stream: ext://sys.stderr
    formatter: basic
  audit_file:
    class: logging.FileHandler
    filename: bot_log.logging
    encoding: utf-8
    formatter: basic
formatters:
  basic:
    style: "{"
    format: "{asctime:s} {levelname:s}: {name:s}: {message:s}"
    datefmt: "%Y-%m-%d %H:%M:%S"
loggers:
  verbose:
    handlers: [console]
    level: DEBUG
    propagate: False
  audit:
    handlers: [audit_file]
    level: DEBUG
root:
  handlers: [console]
  level: INFO

How I now load and configure the logging in my application:

if __name__ == "__main__":
    with LoggingConfig():
        with TradeHub() as th:
            th.run()

Upvotes: 1

Views: 1411

Answers (1)

hoefling
hoefling

Reputation: 66311

requests itself does not log anything, all the logging is emitted from the urllib3 package. Just add a configuration section for the requests.packages.urllib3 logger:

loggers:
  verbose:
    handlers: [console]
    level: DEBUG
    propagate: False
  audit:
    handlers: [audit_file]
    level: DEBUG
  requests.packages.urllib3:
    propagate: True
    level: DEBUG
root:
  handlers: [console]
  level: INFO

Although afaik all the urllib3's logging output has DEBUG level, so you won't catch anything as long as your root logger has INFO level. For example, this snippet won't print anything (logging.yaml is the config file):

import logging.config
import yaml
import requests


if __name__ == '__main__':
    with open('logging.yaml', 'rt') as f:
        config = yaml.safe_load(f.read())
        logging.config.dictConfig(config)
    requests.get('https://stackoverflow.com')

Change the root logger's level to DEBUG to get the debug output:

2018-02-22 13:52:21 DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): stackoverflow.com
2018-02-22 13:52:21 DEBUG: urllib3.connectionpool: https://stackoverflow.com:443 "GET / HTTP/1.1" 200 36997

Upvotes: 1

Related Questions