CODEWITHSUNDEEP
pythonloggingconfiguration
Nupur
Nupur

Reputation: 1081

Multiple instances and multiple destinations

I have multiple Python modules, each of which has separate log config files. I am using Yaml, so I just do a 

    log_config_dict=yaml.load(open(config_file1, 'r'))
    logging.config.dictConfig(log_config_dict)             
    self.main_logger=logging.getLogger('Main_Logger')

In another module, I have something like

   log_config_dict=yaml.load(open(config_file2, 'r')) 
   logging.config.dictConfig(log_config_dict)       
   self.main_logger=logging.getLogger('Poller_Main_Logger')

The 2 loggers are writing to separate log files. Then in the code for each separate module, I do logging as -

     self.main_logger.info(log_str)

However this is not working as expected. Say I do logging from module1,then from module2, then from module1 again, the log messages are either written to module2's destination, or not written at all.

Any idea what is happening? Is the problem that each time I do a dictConfig call, previous loggers are disabled? Is there any way around this?

Below - one of the log config files

version: 1
formatters:
  default_formatter:
    format: '%(asctime)s : %(levelname)s : %(message)s'
    datefmt: '%d-%b-%Y %H:%M:%S'
  plain_formatter:
    format: '%(message)s'
handlers:  
  console_default_handler:
    class: logging.StreamHandler
    level: INFO
    formatter: default_formatter
    stream: ext://sys.stdout  
  console_error_handler:
    class: logging.StreamHandler
    level: WARNING
    formatter: default_formatter
    stream: ext://sys.stderr  
  logfile_handler:    
    class: logging.FileHandler
    filename: logger.txt
    mode: a
    formatter: default_formatter
    level: DEBUG    
  errfile_handler:    
    class: logging.FileHandler
    filename: error.txt
    mode: a
    formatter: default_formatter
    level: WARNING    
  plain_handler:
    class: logging.StreamHandler
    level: DEBUG
    formatter: plain_formatter
    stream: ext://sys.stdout
loggers:
  Poller_Main_Logger:
    level: DEBUG
    handlers: [console_default_handler,logfile_handler]
    propagate: no
  Plain_Logger:
    level: DEBUG
    handlers: [plain_handler]
    propagate: no
  Error_Logger:
    level: WARNING
    handlers: [errfile_handler,console_error_handler,logfile_handler]
    propagate: no
root:
  level: INFO
  handlers: [console_default_handler]

Upvotes: 1

Views: 416

Answers (1)

Silas Ray
Silas Ray

Reputation: 26150

logging doesn't support the usage pattern you want here. By default, rerunning logging.config.dictConfig() blows away all existing loggers, handlers, and formatters. There are the incremental and disable_existing_loggers options available for use in the config dict, but using incremental, you can't load in new handlers or formatters. You either need to combine you configuration in to a single file for your whole program, or use the mechanisms provided in the module to manually construct and add formatters and handlers to loggers.

Upvotes: 1

Related Questions