CODEWITHSUNDEEP
pythonpython-3.xloggingini
edo101
edo101

Reputation: 639

Python Log File is not created unless basicConfig is called on top before any functions

I have a script that processes csvs and load them to database. My intern mentor wanted us to use log file to capture what's going on and he wanted it to be flexible so one can use a config.ini file to edit where they want the log file to be created. As a result I did just that, using a config file that use key value pairs in a dict that i can extract the path to the log file from. These are excepts from my code where log file is created and used:

dirconfig_file = r"C:\Users\sys_nsgprobeingestio\Documents\dozie\odfs\venv\odfs_tester_history_dirs.ini"
start_time = datetime.now()

def process_dirconfig_file(config_file_from_sysarg):
    try:
        if Path.is_file(dirconfig_file_Pobj):
            parseddict = {}
            configsects_set = set()
            for sect in config.sections():
                configsects_set.add(sect)
                for k, v in config.items(sect):
                    # print('{} = {}'.format(k, v))
                    parseddict[k] = v
            print(parseddict)
            try:
                if ("log_dir" not in parseddict or parseddict["log_dir"] == "" or "log_dir" not in configsects_set):
                    raise Exception(f"Error: Your config file is missing 'logfile path' or properly formatted [log_file] section for this script to run. Please edit config file to include logfile path to capture errors")
    except Exception as e:
        #raise Exception(e)
        logging.exception(e)
        print(e)

parse_dict = process_dirconfig_file(dirconfig_file)
logfilepath = parse_dict["log_dir"]
log_file_name = start_time.strftime(logfilepath)
print(log_file_name)
logging.basicConfig(
    filename=log_file_name,
    level=logging.DEBUG,
    format='[Probe Data Quality] %(asctime)s - %(name)s %(levelname)-7.7s %(message)s'
    # can you explain this Tenzin?
)

if __name__ == '__main__':
    
    try:
        startTime = datetime.now()
        db_instance = dbhandler(parse_dict["db_string"])
        odfs_tabletest_dict = db_instance['odfs_tester_history_files']
        odf_history_from_csv_to_dbtable(db_instance)
        #print("test exception")
        print(datetime.now() - startTime)
    except Exception  as e:
        logging.exception(e)
        print(e)

Doing this, no file is created. The script runs with no errors but no log file is created. I've tried several things including using a hardcoded log file name, instead of calling it from the config file but it didn't work

The only thing that works is when the log file is created up top before any method. Why is this?

Upvotes: 0

Views: 270

Answers (1)

C.Nivs
C.Nivs

Reputation: 13106

When you are calling your process_dirconfig_file function, the logging configuration has not been set yet, so no file could have been created. The script executes top to bottom. It would be similar to doing something like this:

import sys

# default logging points to stdout/stderr kind of like this
my_logger = sys.stdout

my_logger.write("Something")

# Then you've pointed logging to a file
my_logger = open("some_file.log", 'w')

my_logger.write("Something else")

Only Something else would be written to our some_file.log, because my_logger pointed somewhere else beforehand.

Much the same is happening here. By default, the logging.<debug/info> functions do nothing because logging won't do anything with them without additional configuration. logging.error, logging.warning, and logging.exception will always at least write to stdout out of the box.

Also, I don't think the inner try is valid Python, you need a matching except. And I wouldn't just print an exception raised by that function, I'd probably raise and have the program crash:

def process_dirconfig_file(config_file_from_sysarg):
    try:
        # Don't use logging.<anything> yet
        ~snip~
    except Exception as e:
        # Just raise or don't use try/except at all until
        # you have a better idea of what you want to do in this circumstance
        raise

Especially since you are trying to use the logger while validating that its configuration is correct.

The fix? Don't use the logger until after you've determined it's ready.

Upvotes: 1

Related Questions