CODEWITHSUNDEEP
pythonpython-2.7
Amelio Vazquez-Reina
Amelio Vazquez-Reina

Reputation: 96284

Module imports from relative folders with regular execution

Say I have the following structure:

app/
   __init__.py
   mod.py

   pkg/
      __init__.py
      submod.py

where the module submod has a relative import to mod, i.e.:

from .. import mod

I understand that if I want to execute submod as a script, I can do the following from app/

python -m pkg.submod

But I would like submod.py to be an executable module that I can call from anywhere in the system with

python /path/to/submod.py

I thought that PEP-366 fixed this, i.e. I thought that adding the following boilerplate code before I do any relative imports in submod:

if __name__ == "__main__" and __package__ is None:
    __package__ = "app.pkg"

I could then go back to the regular python /path/to/submod.py. However, when I do that I get:

SystemError: Parent module 'app' not loaded, cannot perform relative import

Why?

Finally I understand that one solution is to manipulate sys.path in submod so that it can see mod1 and then do the regular import mod1 and avoid relative imports. But as this question shows, this is dangerous because any changes to sys.path in one module propagate to everything else, so in general it is not a good idea to tamper with sys.path.

Is there any way to have either:

or

?

Upvotes: 2

Views: 274

Answers (2)

abarnert
abarnert

Reputation: 365717

As PEP 366 explicitly says right after the boilerplate that you copied:

Note that this boilerplate is sufficient only if the top level package is already accessible via sys.path. Additional code that manipulates sys.path would be needed in order for direct execution to work without the top level package already being importable.

So, assuming your code looks like this:

if __name__ == "__main__" and __package__ is None:
    __package__ = "app.pkg"
from .. import mod

You should get an exception like this:

SystemError: Parent module 'app' not loaded, cannot perform relative import

As the error message implies, you need to get app imported somehow before you can do any relative imports. That obviously means you can't relative-import app, so you need to absolute-import it. There are various hacky ways you could do this, or you could do what the PEP suggests and munge sys.path with all of the problems that implies. But otherwise, you can't do this.

This is just one of multiple reasons that trying to run a script from the middle of a package is hard. And that's intentional; as explained in PEP 3122:

Guido views running scripts within a package as an anti-pattern.

There are two ways to accomplish what you want. The first, you've already got working: just run the module as a module instead of a script. (And making that work is what PEP 366 was primarily for.) The other is to split the module and script into two pieces. Which you can do trivially just by writing a wrapper.

For example, let's say submod.py looks like this silly toy example:

if __name__ == "__main__" and __package__ is None:
    __package__ = "app.pkg"
from .. import mod

def main(argv):
    print(argv)

if __name__ == '__main__':
    import sys
    main(sys.argv)

Create a new file, say, sub.py, as a sibling to the package:

import sys
import app.pkg.submod
app.pkg.submod.main(sys.argv)

Now, you can run sub.py from anywhere on the system. As sys.path says:

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter.

Since that directory is also the directory app is in, you're guaranteed to be able to absolute-import app.pkg.submod, which means that submod will be able to relative-import whatever it wants.

Upvotes: 3

user2357112
user2357112

Reputation: 280778

The package isn't pkg2; it's app.pkg2. Set __package__ accordingly:

if __name__ = '__main__' and __package__ is None:
    __package__ = 'app.pkg2'

If that's not the problem, then app probably isn't on the path. The best solution I know of is to put it on the path (by moving app or putting its current directory in PYTHONPATH), but if you're looking for a way to let modules in app see each other without being visible to external modules, I don't know of anything. It seems like it could be useful.

Upvotes: 1

Related Questions