How to import a package module from inside a Flask route?

Question

I am migrating an app I have written to the standard Python package format. As I have my app's modules divided over several directories, I used to import these by inserting their respective paths to the sys.path, but this is no longer an option as these modules also import their own respective dependencies from inside the package.

I am trying to load a set of modules from inside a route (@app.route) in Flask in a separate file.

@app.route("/interface")
def interface():
    moduleName = "Olympus.modules.%s.%s" % (category, module)
    print "Importing:", moduleName
    importedModule = __import__(moduleName)

(I have removed extranuous code in this function.) This function however, only yields ImportErrors, telling me "there is no module named ". There are __init__.py files in every (sub)directory and these modules import effortlessly elsewhere. Importing the modules from the containing directory's __init__.py results in the modules and their siblings (eg. all the scripts in the same category) being accessible from the script, but this requires all me tot hardcode all the modules in the init file.

The routes are located in a separate file as suggested by Flask (http://flask.pocoo.org/docs/patterns/packages/) which is imported by the init file of the app. When __name__ is printed it yields the proper, packaged name. The modules can also be imported individually at the top of the file, with no ill effects. Printing pkgutils.itermodules as suggested here inside the function yields an empty list.

What am I doing wrong SO? How do I import these modules into this Flask route?

EDIT: The code in question can be found here: https://github.com/HAN-Olympus/Olympus/tree/master/webapp EDIT: Here is the directory structure, I have pruned it somewhat, but it is still rather extensive.

.
├── Olympus
│   ├──__init__.py
│   ├── core
│   │   ├── Core.py
│   │   ├── __init__.py
│   │   ├── ModuleLoader.py
│   │   ├── Worker.py
│   ├── createMongoDatabase.js
│   ├── default.conf
│   ├── __init__.py
│   ├── lib
│   │   ├── Article.py
│   │   ├── Chemical.py
│   │   ├── Collection.py
│   │   ├── Config.py
│   │   ├── Controls.py
│   │   ├── Gene.py
│   │   ├── __init__.py
│   │   ├── ...
│   │   ├── TOXNETResult.py
│   ├── modules
│   │   ├── acquisition
│   │   │   ├── AcquisitionModule.py
│   │   │   ├── GeneOntology.py
│   │   │   ├── __init__.py
│   │   │   ├── PubMed.py
│   │   │   ├── TOXNET.py
│   │   │   ├── UniProt.py
│   │   │   ├── WormBase.py
│   │   ├── __init__.py
│   │   ├── interface
│   │   │   ├── __init__.py
│   │   │   ├── InterfaceModule.py
│   │   │   ├── LaTeX.py
│   │   │   ├── PlainHTML.py
│   │   │   ├── StyledHTML.py
│   │   ├── interpretation
│   │   │   ├── __init__.py
│   │   │   ├── InterpretationModule.py
│   │   │   ├── Sort.py
│   │   │   ├── Tail.py
│   │   └── visualization
│   │       ├── FancyTable.py
│   │       ├── __init__.py
│   │       ├── Table.py
│   │       ├── VisualizationModule.py
│   ├── olympus.conf
│   ├── README.md
│   ├── requirements.txt
│   └── webapp
│       ├── __init__.py
│       ├── routes.py
│       ├── start.py
│       ├── svglib.py
│       └── templates
│           ├── ...

Answer 1

Maintain all modules as standard modules

Keep it simple and install your modules to import into your Flask environment.

Make all the modules complete (incl. setup.py)

To make your modules installable, they must include setup.py installation script. Check documentation, how to do this.

Installing modules into Flask app

From your Flask app directory, you can install other modules by:

$ pip install ../other_module_dir

and do so repeatedly.

Installing modules in "edit" mode

If you want to have these installed in the way, that you use the latest version, you can use "edit" installation mode

$ pip install -e ../other_module_dir

With this approach, if you modify code in your ../other_module_dir project, next time it will get imported in updated version.

Automate installation of modules using requirements.txt

To automate installations, you can finally dump all the dependencies into file (usually) named requirements.txt:

$ pip freeze > requiremetns.txt

edit it (remove, what does not belong there, and finally you may install all dependent modules in one step:

$ pip install -r requirements.txt

Conclusion

The most demanding (but not so difficult) is to make your modules complete with setup.py. Once you manage this (and this skill will pay back many times later on), your whole scenario of import will become trivial, just following names of your modules you import and have them available (I recommend using absolute imports).

Note: Use virtualenv for this kind of work, it is not required, but it will make your work much simpler.