CODEWITHSUNDEEP
pythonpython-2.7
Chet
Chet

Reputation: 19889

Python: relative imports without packages or modules

After trying to deal with relative imports and reading the many StackOverflow posts about it, I'm starting to realize this is way more complicated than it needs to be.

No one else is using this code but me. I'm not creating a tool or an api, so I see no need to create a "package" or a "module". I just want to organize my code into folders, as opposed to having a 50 scripts all in the same directory. It's just ridiculous.

Basically, just want to have three folders and be able to import scripts from wherever I want. One folder that contains some scripts with utility functions. Another folder has the majority of the code, and another folder that contains some experiments (I'm doing machine learning research). 

/project/code/
/project/utils/
/project/experiments/

All I need to do is import python files between folders.

One solution I have found is putting __init__.py files in each directory including the root directory where the folders live. But I then need to run my experiments in the parent of that directory.

/experiment1.py
/experiment2.py
/project/__init__.py
/project/code/__init__.py
/project/utils/__init__.py

So the above works. But then I have two problems. My experiments don't live in the same folder as the code. This is just annoying, but I guess I can live with it. But the bigger problem is that I can't put my experiments different folders:

/experiments/experiment1/something.py
/experiments/experiment2/something_else.py
/project/__init__.py
/project/code/__init__.py
/project/utils/__init__.py

I suppose I could symlink the project directory into each experiment folder, but thats ridiculous.

The other way of doing it is treating everything like a module:

/project/__init__.py
/project/code/__init__.py
/project/utils/__init__.py
/project/experiments/__init__.py
/project/experiments/experiment1/something.py
/project/experiments/experiment2/something_else.py

But then I have to run my experiments with python -m project.experiments.experiment1.something which just seems odd to me.

A solution I have found thus far is:

import imp
import os
currentDir = os.path.dirname(__file__)
filename = os.path.join(currentDir, '../../utils/helpful.py')
helpful = imp.load_source('helpful',filename)

This works, but it is tedious and ugly though. I tried creating a script to handle this for me, but then the os.path.dirname(__file__) is wrong.

Surely someone has been in my position trying to organize their python scripts in folders rather than having them all flat in a directory.

Is there a good, simple solution to this problem, or will I have to resort to one of the above?

Upvotes: 1

Views: 475

Answers (2)

Chet
Chet

Reputation: 19889

So I found this tutorial very useful.

The solution I have come up with is as follows:

project/
    .gitignore
    setup.py
    README.rst
    MANIFEST.in
    code/
        __init__.py
        something.py
        tests/
            __init__.py
            tests.py
        utils/
            __init__.py
            utils.py
    experiments/
        experiment1/
            data.json
            experiment1.py
        experiment2/
            data.json
            experiment2.py

Then I run python setup.py develop in order symlink my code so I can import it anywhere else (you can unlink with python setup.py develop --uninstall).

I still haven't decided whether I like my experiments to live inside the project folder or outside. I don't think it really matters since this code is only for my personal use. But I suppose it would be proper for it to live outside...

Upvotes: 0

aacanakin
aacanakin

Reputation: 2913

Running python files as a module seems also odd to me. You can put your experiments in a folder still by putting a main.py file on the root directory. So the folder tree would be the following;

/project/experiments
/project/code
/project/utils
/project/main.py

Call your experiments on your main.py file and make your imports on main.py file. The __init__.pys should also be inside each folder.

By this way, you won't need to run the py files as a python module. Also, the project will have a single entry point which is very useful in many cases.

Upvotes: 2

Related Questions