How to read properties file in python

Question

I have a property file called Configuration.properties containing:

path=/usr/bin
db=mysql
data_path=/temp

I need to read this file and use the variable such as path, db, and data_path in my subsequent scripts. Can I do this using configParser or simply reading the file and getting the value.

Thanks in Advance.

Answer 1

Keeping it simple. Create a properties file of name=value pairs and save it as filename.py. Then all you have to do is import it and reference the properties as name.value. Here's how...

My config file for an Oracle connection is called config.py and looks like this.

username = "username"
password = "password"
dsn = "localhost/xepdb1"
port = 1521
encoding = "UTF-8"

So, in the Python all I have to do is...

import cx_Oracle
import config

conn = None
cursor = None

try:

    print("connecting...");

    conn = cx_Oracle.connect(config.username, config.password, config.dsn, encoding=config.encoding)

    cursor = conn.cursor()

    print("executing...");

    # don't put a ';' at the end of SQL statements!
    cursor.execute("SELECT * FROM employees")

    print("dumping...");

    for row in cursor:
        print(row[0])

except cx_Oracle.Error as error:
    print(error)
finally:
    print("finally...")
    if cursor:
        cursor.close()
    if conn:
        conn.close()

Answer 2

One liner for reading an unstructured properties file (no sections) while ignoring comments:

with open(props_file_path, "r", encoding="utf-8") as f:
    props = {e[0]: e[1] for e in [line.split('#')[0].strip().split('=') for line in f] if len(e) == 2}

Answer 3

If you need to read all values from a section in properties file in a simple manner:

Your config.properties file layout :

[SECTION_NAME]  
key1 = value1  
key2 = value2  

You code:

   import configparser

   config = configparser.RawConfigParser()
   config.read('path_to_config.properties file')

   details_dict = dict(config.items('SECTION_NAME'))

This will give you a dictionary where keys are same as in config file and their corresponding values.

details_dict is :

{'key1':'value1', 'key2':'value2'}

Now to get key1's value : details_dict['key1']

Putting it all in a method which reads that section from config file only once(the first time the method is called during a program run).

def get_config_dict():
    if not hasattr(get_config_dict, 'config_dict'):
        get_config_dict.config_dict = dict(config.items('SECTION_NAME'))
    return get_config_dict.config_dict

Now call the above function and get the required key's value :

config_details = get_config_dict()
key_1_value = config_details['key1'] 

-------------------------------------------------------------

Extending the approach mentioned above, reading section by section automatically and then accessing by section name followed by key name.

def get_config_section():
    if not hasattr(get_config_section, 'section_dict'):
        get_config_section.section_dict = dict()

        for section in config.sections():
            get_config_section.section_dict[section] = 
                             dict(config.items(section))

    return get_config_section.section_dict

To access:

config_dict = get_config_section()

port = config_dict['DB']['port'] 

(here 'DB' is a section name in config file and 'port' is a key under section 'DB'.)

Answer 4

If you want to read a proeprties file in python, my first suggestion, which I am myself not following because I like Visual Code too much ...

Is run your python on Jython. And once you run python on Jython, you can trivially open a java util input stream to your data file. And using the java.utl.Properties you can call the load() api, and you are set to go. So my recommendation is, do what is easiest, just start using the java runtime environment and jython along with it.

By the way, I am of course using jython to run python. No questions on that.

But what I am not doing is using jython to debug python... sadly! The problem for me is that I use microsft visual code for writing pythong, and then yeah ... then I am stuck to the my normal python installation. Not an ideal world!

If this would be your situation. Then you can go to plan (b) ... Try to not use JDK libraries and find an alternative else where.

So here is sugestion. This is a library that I found out and and that I am using for the effect of reading properties files. https://pypi.python.org/pypi/jproperties/1.0.1#downloads

from jproperties import Properties
with open("foobar.properties", "r+b") as f:
    p = Properties()
    p.load(f, "utf-8")

    # Do stuff with the p object...

    f.truncate(0)
    p.store(f, encoding="utf-8")

So in the above code quote, you seen how they open properties file. Wipe it out, and write back again the properties into the file.

You tread the properties object as dictionary object. And then you do stuff like:

myPropertiesTuple = propertiesObjec[myPropertyKey]

Be careful. When you use the above api and get a value for a key, that value is a PropertiesTuple. It is a pair of (value,metadata). So the value you would be hunting for you get it from myPropertiesTuple[0]. Other then that, just read the documentation on the page the library works well.

I am using approach (b). If the advantage of using the java libraries at some point outweighs the advantage of being stuck to native python language simply so that I may debug on visual code.

I will I hear beat drop support for pure python runtime and comrpomise the code with hard dependencies jython runtime / java libararies. So far there is no need for it.

So, blue bill or the red pill?

Answer 5

For a configuration file with no section headers, surrounded by [] - you will find the ConfigParser.NoSectionError exception is thrown. There are workarounds to this by inserting a 'fake' section header - as demonstrated in this answer.

In the event that the file is simple, as mentioned in pcalcao's answer, you can perform some string manipulation to extract the values.

Here is a code snippet which returns a dictionary of key-value pairs for each of the elements in your config file.

separator = "="
keys = {}

# I named your file conf and stored it 
# in the same directory as the script

with open('conf') as f:

    for line in f:
        if separator in line:

            # Find the name and value by splitting the string
            name, value = line.split(separator, 1)

            # Assign key value pair to dict
            # strip() removes white space from the ends of strings
            keys[name.strip()] = value.strip()

print(keys)

Answer 6

I like the current answer. AND... I feel like there is a cleaner way to do it in the "Real World". Using the section header feature is a MUST if you are doing a project of any size or scale, particularly in the realm of "multiple" environments. I wanted to put it in here with well formatted copy-able code, using a robust real world example. This is running in Ubuntu 14, but works cross platform:

Real World Simple Example

An 'Environment-based' configuration
Setup Example (Terminal):

cd ~/my/cool/project touch local.properties touch environ.properties ls -la ~/my/cool/project -rwx------ 1 www-data www-data 0 Jan 24 23:37 local.properties -rwx------ 1 www-data www-data 0 Jan 24 23:37 environ.properties

Set good permissions

>> chmod 644 local.properties
>> chmod 644 env.properties
>> ls -la
-rwxr--r-- 1 www-data www-data  0 Jan 24 23:37 local.properties
-rwxr--r-- 1 www-data www-data  0 Jan 24 23:37 environ.properties

Edit your properties files.

FILE 1: local.properties

This is YOUR properties file, local to your machine and workspace, and contains sensitive data, don't push to version control!!!

[global]
relPath=local/path/to/images              
filefilters=(.jpg)|(.png)

[dev.mysql]
dbPwd=localpwd
dbUser=localrootuser

[prod.mysql]
dbPwd=5tR0ngpwD!@#
dbUser=serverRootUser

[branch]
# change this to point the script at a specific environment
env=dev

FILE 2: environ.properties

This properties file is shared by everybody, changes are pushed to version control

#----------------------------------------------------
# Dev Environment
#----------------------------------------------------

[dev.mysql]
dbUrl=localhost
dbName=db

[dev.ftp]
site=localhost
uploaddir=http://localhost/www/public/images

[dev.cdn]
url=http://localhost/cdn/www/images


#----------------------------------------------------
# Prod Environment
#----------------------------------------------------
[prod.mysql]
dbUrl=http://yoursite.com:80
dbName=db

[prod.ftp]
site=ftp.yoursite.com:22
uploaddir=/www/public/

[prod.cdn]
url=http://s3.amazon.com/your/images/

Python File: readCfg.py

This script is a re-usable snippet to load a list of configuration files import ConfigParser import os

# a simple function to read an array of configuration files into a config object
def read_config(cfg_files):
    if(cfg_files != None):
        config = ConfigParser.RawConfigParser()

        # merges all files into a single config
        for i, cfg_file in enumerate(cfg_files):
            if(os.path.exists(cfg_file)):
                config.read(cfg_file)

        return config

Python File: yourCoolProgram.py

This program will import the file above, and call the 'read_config' method

from readCfg import read_config

#merge all into one config dictionary
config      = read_config(['local.properties', 'environ.properties'])

if(config == None):
    return

# get the current branch (from local.properties)
env      = config.get('branch','env')        

# proceed to point everything at the 'branched' resources
dbUrl       = config.get(env+'.mysql','dbUrl')
dbUser      = config.get(env+'.mysql','dbUser')
dbPwd       = config.get(env+'.mysql','dbPwd')
dbName      = config.get(env+'.mysql','dbName')

# global values
relPath = config.get('global','relPath')
filefilterList = config.get('global','filefilters').split('|')

print "files are: ", fileFilterList, "relative dir is: ", relPath
print "branch is: ", env, " sensitive data: ", dbUser, dbPwd

Conclusion

Given the above configuration, you can now have a script which completely changes environments by changing the [branch]env value in 'local.properties'. And this is all based on good configuration principles! Yaay!

Answer 7

Yes, yes you can.

ConfigParser (https://docs.python.org/2/library/configparser.html) will give you a nice little structure to grab the values from out of the box, where doing by hand will require some string splitting, but for a simple format file, it's no big deal.

Is the question "How do I read this file?".