CODEWITHSUNDEEP
pythonlogging
Homunculus Reticulli
Homunculus Reticulli

Reputation: 68476

Python: correct way to log to the same file from different modules

I have written a bunch of scripts over time and I am in the process of refatoring the scripts to keep the code DRY. I am currently using something along these lines in the various scripts:

if __name__ == '__main__':
    logger = logging.getLogger('dbinit')
    hdlr = logging.FileHandler('/var/logs/tmp/foo.log')
    formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
    hdlr.setFormatter(formatter)
    logger.addHandler(hdlr) 
    logger.setLevel(logging.WARNING)

Rather than repeating this in every script (i.e. "module"), I would like to have this logger initialisation done one somewhere and accessed by the various scripts (Hmm, maybe wrap in a singleton class?).

If I can't do that (i.e. put the logger initialisation code in one core module), I assume that by using the same log file name in the logging.FileHandler() call, the various scripts will write to the same file.

Is this assumption correct?

Last but not the least, what is the best practise (i.e. Pythonic) way to solve this problem?

Upvotes: 12

Views: 18570

Answers (3)

Aravinth s.r
Aravinth s.r

Reputation: 21

create a function in your python module like below:

def createLogHandler(job_name,log_file):
    logger = logging.getLogger(job_name)
    ## create a file handler ##
    handler = logging.FileHandler(log_file)
    ## create a logging format ##
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    return logger

Now call the function in you program like this:

job_name = 'dbinit'
log_file = '/var/logs/tmp/foo.log'
logger = createLogHandler(job_name ,log_file )
logger.info('Logger has been created')

Upvotes: 2

Phani
Phani

Reputation: 3325

Given that you are using if __name__ == __main__, I am assuming that these scripts would be running as different processes. If that's the case, then you should use a separate configuration file.

This configuration can be in stored in a file in the format as specified in the docs. Then, you can use logging.config.fileConfig to load the file. You can also have the configuration stored in JSON/YAML formats, convert it into a dictionary and load it using logging.config.dictConfig. The latter is the current recommended approach, although I find the former more straight forward. Read this for more information.

The advantages of using a config file approach are many fold:

  1. It keeps the configuration settings separated from your code
  2. It lets non-programmers make changes to configuration, as they are stored in easy to read formats
  3. It also keeps you from repeating yourself as you have already mentioned.

Upvotes: 2

Piotr Dobrogost
Piotr Dobrogost

Reputation: 42465

(...) addHandler() will not add a handler if the handler already exist so having such init code in many places won't hurt.

If I can't do that (i.e. put the logger initialisation code in one core module)

You can to this and you should do this if you want to ensure initialization code is run once.

Upvotes: -1

Related Questions