CODEWITHSUNDEEP
pythonimportmoduleconfig
loqqi
loqqi

Reputation: 81

one config file for entire python app

I'm somewhat new to Python (but not at all to programming, already fluent in Perl, PHP, & Ruby). I'm running into a major difference between all my previous languages and Python: Mainly that I can easily read in data from another file into an entire project. there are two major examples of this that I would like to solve:

1) I would like to have a settings/config file (in YAML) that gets read in and parsed in a config.py file. I then want a resulting dict() to be accessible to all my other files in the project. I have figured out how to do this by from lib.project.config import cfg but that means that for each page that is importing the configs the system has to REparse the yaml. That seems just silly to me. Is there no way to have that file parsed just once and then have the results accessible to any other file in my project?

2) I would like to import a database.py file that then looks at my configs to see if we need to import a sqlite3, mysql, or postgresql version of a database class. Again, I can manage this by putting the logic directly in each page that I need the database class for. But I hate having to paste code like

if cfg.get('db_type') == 'sqlite':
    from lib.project.databases.sqlite3 import database
elif cfg.get('db_type') == 'mysql':
    from lib.project.databases.mysql import database

at the top of each file that needs the database class. I'd much rather just add:

import lib.project.database

Any help would be very much appreciated. I've done a good deal of googling and SO searches and not found my answers. Hopefuly one of you wiz's out there can help.

Thanks.

UPDATE: The reason I'm doing things this way (for #2) is because I'm also trying to make other classes inherit the database class. So lib/project/databases/sqlite.py is a definition of a database class. And so is lib/project/databases/mysql.py. The idea being that after this import is complete I can import classes like the users class and define it like so:

class user(database):
    ...

And thus inherit all of the structure and methods of the database class. Your suggestions to simply create an instance based on the sqlite/mysql logic/decision and then pass that where it needs to be is a good solution for that. But I need a bit more... Ideas?

Upvotes: 5

Views: 3283

Answers (2)

gnr
gnr

Reputation: 2414

1) you're all set - the file will only be read once.

2) Agreed you don't want to copy any paste code like that - if you want to use 1 instance of the database throughout your project, you'd do something like:

import lib.project

db = lib.project.database

where db is just a local variable used to access your already created database. You would instantiate your database as database (resolving as you've done with the if/elif code whether to use sqlite3 or mysql) in lib/project/databases.py and then in your lib/project/__init__.py file you would do a from .databases import database.

if you want to use multiple databases (of the same sqlite3/mysql type) throughout your project, you would resolve which constructor Database is bound to (or inherits from) in databases.py:

from lib.project.Config import config
if config.db_type == 'sqlite' :
    import lib.project.databases.Sqlite as Db
elif config.db_type == 'mysql':
    import lib.project.databases.Mysql as Db

class Database(Db):
    '''docs'''

Upvotes: 2

loqqi
loqqi

Reputation: 81

Thank you all for your help in understanding more about Python and how to get what I'm looking for done.

Here is what I ended up doing:

1) The config file issue: This apparently was mostly solved to begin with. I confirmed what everyone was saying: that you can "import" a file as many times as you like, but it is only ever parsed/processed/compiled once. As for making it reachable to any file that needs it: Config class:

import os
import yaml

class Config:

    def __init__(self):
        self.path = os.getcwd()
        stream = open(self.path+"/conf/config.yaml", 'r')
        data = yaml.load(stream)
        config_keys = data.keys()
        for k in config_keys:
            setattr(self, k, data.get(k))
        if (os.path.isfile(self.path+"/conf/config-override.yaml") ):
            stream = open(self.path+"/conf/config-override.yaml", 'r')
            data = yaml.load(stream)
            config_keys = data.keys()
            for k in config_keys:
                setattr(self, k, data.get(k))

config = Config()

And then any file that wants to use it:

from lib.project.Config import config

This is all working swimmingly so far.

2) Dynamic database type for Database class: I just altered my overall design a tiny bit to make a Database class (mostly empty) inherit from either the Sqlite or Mysql classes (both custom builds which are wrappers to existing Sqlite3 and mysql-connector classes). This way there is always a solid Database class to inherit from, I only load in what files I need to, and it's all defined by my config file. Example: Database class:

from lib.project.Config import config
if config.db_type == 'sqlite' :
    from lib.project.databases.Sqlite import Sqlite
elif config.db_type == 'mysql':
    from lib.project.databases.Mysql import Mysql

class Database(Sqlite if config.db_type == 'sqlite' else Mysql):
''' documentation '''

I'd still love to hear people's feedback on this code/method. As I said, I'm still newish to Python and could still be missing something.

Thanks again everyone.

Upvotes: 3

Related Questions