How to access an editable configuration file once a Python package has been installed via setuptools or distutils?

Question

How do I allow a package installed with 'python setup.py install' to access external configuration files?

I am currently trying to use distutils/setuptools to install a Python package for RHEL. This Python package was originally written in such a way that it operated as a set of scripts, which needed filesystem access to a variety of configuration files, logs, etc. Now that I am trying to create an installable package out of this setup, I'm having difficulties interacting with the filesystem for the necessary files.

These files should be editable. In addition to the base question, what are the best practices for this type of behavior? I.e. what are the best practices for managing the configuration of Python packages that are installed via setuptools or distutils, and for these packages to interact with the filesystem. I assume there are two halves of this coin: configuring the locations for the files on the install side, and accessing the information on the code implementation side. Any advice for optimizing both of these aspects would be greatly appreciated.

Additional Information:

I essentially need to access the "config" directory, once the package is installed. The location of the 'config' directory can be anywhere (that the user has privileges to) on the filesystem, and should, in itself, be configurable before installing the package. The project looks like this:

• project/
  • main/
    • __init__.py
    • script1.py
    • script2.py
    • service.py (Flask web service runner)
    • ...other python files that interact with above
  • config/
    • sub-directory/
      • sub-directory/
        • config_file.ini
    • sub-directory/
      • ...
    • ...
  • logs/
    • log_file.log

Answer 1

You can install data files with the data_files option in distutils. However, distutils/setuptools are quite limited in scope in what you can do with handling config files. You can only specify a relative or absolute path where these files will be installed, there isn't an option to make this installation location user configureable.

Best practices for handling installation and reading of config files? I don't think there is anything that is canonically regarded as "best". distutils is quite limited in what it can do, and therefore won't meet many use cases. You are essentially left to handling config files in whatever way meets your requirements.

If you want to configure the location of the config files before installation, you need something that provides more than distutils does. A few possible ways to handle this:

1. Add code to your setup.py script to handle this. setup.py can handle arbitrary Python code, so it can be extended beyond just the required setup() call to do whatever you need. However, doing so is considered unexpected and non-standard. In the days before tools like setuptools and pip, packages would sometimes have an interactive prompt would ask the user for install input. But this would break when that package was installed by an automated installed such as pip or easy_install.

2. Write your own installer. This handles the details of where to install your config files and installs your Python package under-the-hood.

3. In a linux environment, the Buildout tool can be helpful for this. This would require the user to install Buildout and then edit a buildout.cfg configuration file with the config file location. It can be a helpful way to manage application configuration, but it does require the person doing the installation to have Buildout installed and edit a bulidout-based config file. An example of this is the Plone project, which has a complex and configurable install process and Buildout is used to manage this process.