CODEWITHSUNDEEP
pythonconfigparser
CoderOfTheNight
CoderOfTheNight

Reputation: 1122

Configparser Occasionally Returns Empty Result

I am reading configurations from an ini file using Python configparser. 95% of the time, it works perfectly. Without any changes to the ini file (e.g., in the middle of a regression test), configparser will start returning empty results, and then of course have key errors as it searches for nested configs.

This suggests it is not a problem with the ini file, since it is not changing, and also because it happens occasionally. Once the problem happens, it continues to happen across all calls to config parser until i kill the program and re-run it.

I'm reading the config as such:

try:
    path = os.path.dirname(os.path.realpath(__file__))
    kml_ini = '/'.join([path, 'kml.ini'])
    config = configparser.ConfigParser()
    config.read(kml_ini)
    db_conn_returnable = config['database']['db_connection'].strip()
except Exception as e:
    print(e)        
    traceback.print_exc(file=sys.stdout)
    pprint.pprint(config.sections())
    pprint.pprint({section: dict(config.items(section)) for section in config.sections()})

I'm getting the error: File "/home/sahmed/anaconda3/envs/kml/lib/python3.6/configparser.py", line 959, in getitem KeyError: 'database'

My ini file looks as such:

[api]
server_debug=False
log_level=info
n_workers=10

[database]
db_connection=http://1.1.1.1:9191
user=admin
pass=whateverman

[cluster]
default_workers=3

My original thought was a thread issue since I've got 8 threads hitting this config file constantly (though it doesnt make sense...we're reading only) so I even put a fasteners block on the read, still no luck. What might cause such a situation?

Upvotes: 4

Views: 2688

Answers (2)

CoderOfTheNight
CoderOfTheNight

Reputation: 1122

I've resolved the issue. The Python logger was pulling too many file descriptors exhausting the max limit (there was a getLogger() inside a frequently called function, not globally) -- so the overall Python process was not able to open the config file.

The general answer here would be that ConfigParser is, of course, exposed to overall system issues where it may not be able to read a file it could read earlier. Not because of the file itself, but because of overall system limitations being reached.

Upvotes: 1

Hampus Larsson
Hampus Larsson

Reputation: 3100

The only way I could replicate the problem with your provided code, was if I forcefully changed the working-directory during the code.

I put together this to test the problem:

from time import sleep
import configparser
import os

def readINI():
    try:
        path = os.path.dirname(os.path.realpath(__file__))
        ini = '/'.join([path, "test.ini"])
        config = configparser.ConfigParser()
        config.read(ini)
        string = config["things"]["abc"]
        print(string)
    except Exception as e:
        print("Error: {}".format(e))

count = 0
while True:
    count +=1
    readINI()
    sleep(1)
    if count % 3 == 0:
        os.chdir("..")
    elif count > 1 and count % 3 == 1:
        os.chdir("./TestFolder")

test.ini is very simple, it just contais two rows:

[things]
abc=123

I have both files in the same folder, and move back and forth between the "root" folder and TestFolder which contains the files. If you raise the exception instead of just printing it, then you will get the same error message.

Please verify that you're not accidentally changing the working directory during code-execution, and see if this indeed is the reason for the problems.

Upvotes: 2

Related Questions