Python3 correct way to import relative or absolute?

Question

I am writing a python module neuralnet. It was working all fine in Python2, but in Python3 imports are failing.

This is my code structure.

neuralnet/
    __init__.py
    train.py         # A wrapper to train (does not define new things)
    neuralnet.py     # Defines the workhorse class neuralnet
    layer/
        __init__.py
        inlayer.py       # Defines input layer class
        hiddenlayer.py

application/         # A seperate application (not part of the package)
   classify.py       # Imports the neuralnet class from neuralnet.py

train.py needs to import neuralnet.py's neuralnet class.

neuralnet.py needs to import layers/inlayer.py etc.

(I prefer relative imports.)

I have a different application (classify.py) which needs to import this module. Where I do...

from neuralnet.neuralnet import neuralnet

I have tried a few ways to import. Either I get an error (mostly arcane like parent is not imported)

1. While running train.py (which is a part of the neuralnet module)

   from . import layer  # In file neuralnet.py
   SystemError: Parent module '' not loaded, cannot perform relative import
   

Or

2. while running classify.py (which is outside the module).

   from layer.inlayers import input_layer   # In file neuralnet.py
   ImportError: No module named 'layer'
   

My imports worked perfectly well for years in Python2. I am wondering what Python3 expects of me? Should I move train.py to outside my module (technically it is not a part of the module)? Please suggest best practice.

Answer 1

Relative import usage has changed from python2 to python3,

The only acceptable syntax for relative imports is from .[module] import name. All import forms not starting with . are interpreted as absolute imports. (PEP 0328)

Python’s -m switch allows running a module as a script. When you ran a module that was located inside a package, relative imports didn’t work correctly. The fix for Python 2.6 adds a __package__ attribute to modules. When this attribute is present, relative imports will be relative to the value of this attribute instead of the __name__ attribute. (PEP 0366)

To your questions:

1. You get SystemError: Parent module '' not loaded, cannot perform relative import or in Python3.5 and forward you would get ImportError: attempted relative import with no known parent package when you run python neuralnet.py and try to import from . import layers as your __package__ would be None and PYTHONPATH will only have current file(and not its parent), hence it can't find layer.

You may either run the module like this:

python -m neuralnet.neuralnet

Here your __package__ will be neuralnet, hence you will be able to import the neuralnet module which is within it.

Or you may do this workaround:

Update the __init__.py in neuralnet package to:

import os
import sys
sys.path.append(os.path.dirname(os.path.realpath(__file__)))

Then run your script neuralnet.py, the above lines will add neuralnet directory to the PYTHONPATH.

2. You get ImportError: No module named 'layer' when layer is not a module that is in your PYTHONPATH so either install it or add it to the PATH using

   import sys
   sys.path.append("/path/to/layer")
   

---

Background:

Part of a message from Guido(author of Python):

... in 2.4, we introduce the leading dot notation for relative import, while still allowing relative import without a leading dot. In 2.5 we can start warning about relative import without a leading dot (although that will undoubtedly get complaints from folks who have code that needs to work with 2.3 still). In 3.0 we can retire ambiguous import.

The use case for multiple dots should be obvious: inside a highly structured package, modules inside one subpackage must have a way to do relative import from another subpackage of the same parent package.

There is the remaining issue of what exactly the syntax would be. I propose to extend the from clause to allow one or more dots before the dotted name, and to make the dotted name optional if at least one leading dot is found. I propose not to change from-less import.

Examples:

  from .foo import bar
  from .foo.bar import xxx
  from . import foobar as barfoo
  from ..foo.bar import *
  from ... import foobar, barfoo

Relavent PEP to read: PEP-328

Answer 2

In Python 3, implicit relative imports are forbidden, see https://www.python.org/dev/peps/pep-0328/ and https://docs.python.org/release/3.0.1/whatsnew/3.0.html#removed-syntax:

The only acceptable syntax for relative imports is from .[module] import name. All import forms not starting with . are interpreted as absolute imports. (PEP 0328)

from .stuff import Stuff is an explicit relative import, which you "should" make use of whenever possible, and must use in Python 3, whenever possible. Head over to https://stackoverflow.com/a/12173406/145400 for a deeper analysis on relative imports.