CODEWITHSUNDEEP
pythonpython-3.xgitpackagesys.path
walfab.dev
walfab.dev

Reputation: 3

Python3 framework with own packages and shared with git: How to handle sys.path and __init__.py files?

I started to build a data processing framework in Python 3. The goal is to develop own packages and write example code to use the packages. The framework will be shared (any maybe developed) with other researchers by git.

What is the best way to handle the search paths on multiple computers and how should the __init__.py files look like in the directories?

I was thinking about adding sys.path relative to the main_directory (Python: Best way to add to sys.path relative to the current running script) with sys.path.append(os.getcwd()) or sys.path.append(".."). I don’t know if this is a good way to import beyond top level on different machines with different absolute paths and operation systems. (The beyond top level package error/problem: beyond top level package error in relative import)

The basic directory structure might be (I will add more packages and processing functions later):

main_directory/
main_directory/__init__.py
main_directory/packages/
main_directory/packages/__init__.py
main_directory/packages/preprocessing/
main_directory/packages/preprocessing/filters.py    #implementation of multiple classes: e.g. fillter1 and filter2
main_directory/packages/preprocessing/__init__.py
main_directory/packages/postprocessing/
main_directory/packages/postprocessing/filters.py   #implementation of multiple classes: e.g. fillterA and filterB
main_directory/packages/postprocessing/__init__.py
main_directory/examples/
main_directory/examples/easy_example.py #file with examples to use the filters
main_directory/your_code/
main_directory/your_code/your_code.py   #file with your code to use the filters

Upvotes: 0

Views: 266

Answers (1)

Eddy Pronk
Eddy Pronk

Reputation: 6705

There is a standard package layout. If you follow that you can deploy your programs without having to touch sys.path.

Use the resources on this website and learn setuptools.

https://packaging.python.org/tutorials/packaging-projects/

Update

During development you can use a virtual environment. First create one:

$ virtualenv venv

Installing your package with -e creates a link to your directory.

$ venv/bin/pip install -e yourproject.git

When you start the python in your virtual environment your imports should work.

$ venv/bin/python
>>> import preprocessing.filters
>>>

Upvotes: 1

Related Questions