CODEWITHSUNDEEP
pythonpython-2.7logging
twfx
twfx

Reputation: 1694

Python logging fail to write into multiple log files

I'd like to automate some tests and log the output of each test into different files, but it seems that all tests outputs are appended into the first log file, instead of separate files. May I know how can I correct this?

What I did is: First, define runthis.cfg which contains all the test config file. Second, execute run.py. It imports class A, for loop and write some log data for each loop

In runthis.cfg

2
prob1.cfg
prob2.cfg

In run.py

from fa import A

def run_it(cfg_data):
    cfg = cfg_data.split('\n')
    runCount = int(cfg[0])

    for i in range(1, runCount+1):        
        print "\n\nRunning " + cfg[i] + "..."
        cA.readProblemInstance(cfg[i])



cA = A() 

if __name__ == '__main__':    
    run_config_file = open('runthis.cfg', 'r')
    cfg_data = ''.join(run_config_file.readlines())
    run_config_file.close()        

    run_it(cfg_data)

In fa.py

from datetime import datetime
import time
import logging

class A():
    def activateLogFile(self, f):
        logging.basicConfig(filename=f, level=logging.INFO)

    def readProblemInstance(self, fn):    
        fn = fn.replace('.', '_')
        fn = fn + '_' + datetime.now().strftime('%Y_%m_%d_%H_%M_%S_%f')
        self.activateLogFile(fn)

        logging.info("%s" %fn)

The output is, in prob1_cfg_2014_04_07_12_39_38_293000,

INFO:root:prob1_cfg_2014_04_07_12_39_38_293000
INFO:root:prob2_cfg_2014_04_07_12_39_38_294000

prob2_cfg_2014_04_07_12_39_38_294000 doesn't exists!

Upvotes: 0

Views: 827

Answers (2)

ebarr
ebarr

Reputation: 7842

@metatoaster has already given a good account of why your code does not log to multiple files, so this answer will only deal with how you log to multiple files given your example.

So the handlers for a a given logger are all stored in .handlers attribute of that logger. This is basically just a list that can be added to with the addHandler method of the logger. In this case your code would become:

# new style classes should inherit from object
class A(object):
    def __init__(self):
        # create a logger for the class and set the log level
        # here we use the root logger
        self.logger = logging.getLogger()
        self.logger.setLevel(logging.INFO)

    def activateLogFile(self, f):
        # create a handler with the name defined by the variable f
        handler = logging.FileHandler(f)
        # add that handler to the logger
        self.logger.addHandler(handler)

    def readProblemInstance(self, fn):    
        fn = fn.replace('.', '_')
        fn = fn + '_' + datetime.now().strftime('%Y_%m_%d_%H_%M_%S_%f')
        self.activateLogFile(fn)
        self.logger.info("%s" %fn)

Upvotes: 1

metatoaster
metatoaster

Reputation: 18898

logging.basicConfig doesn't replace already existing handlers (it does say, "This function does nothing if the root logger already has handlers configured for it."), which was set up by the very first call of that function. What you have to do is remove whatever handler attached to the root logger, and add a new one either by calling basicConfig again or other functions.

Note the following behavior, when you didn't remove the original handler

>>> rootlogger = logging.getLogger()
>>> logging.basicConfig(filename='/tmp/testlog.txt', level=logging.INFO)
>>> rootlogger.handlers
[<logging.FileHandler object at 0xd4a450>]
>>> logging.basicConfig(filename='/tmp/testlog.txt', level=logging.INFO)
>>> rootlogger.handlers
[<logging.FileHandler object at 0xd4a450>]

Note the same object id. Now assign the existing handler to a variable (persist it), then remove, then call logging.basicConfig again.

>>> handler = rootlogger.handlers[0]
>>> rootlogger.removeHandler(handler)
>>> logging.basicConfig(filename='/tmp/testlog.txt', level=logging.INFO)
>>> rootlogger.handlers
[<logging.FileHandler object at 0x1161350>]

Note the different object id.

Upvotes: 2

Related Questions