Python3: module 'x' has no attribute 'y'

Question

Python import statement is confusing the heck out of me. Can someone help me clear this up?

file tree looks like this

root
+- notebook.ipynb
+- lib/
  +- basestation_config.py
  +- config.py
+- config/
  +- valence_pod.json
  +- etc…

in config.py I have:

import json
import os

default_config_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'config'))

def read_filepath(filepath):
    with open(filepath, "r") as read_file:
        return json.load(read_file)

def read(filename):
    filepath = os.path.join(default_config_path, filename) + '.json'
    return read_filepath(filepath)

in basestation_config.py I have:

import config as config
# … a buncha class libraries, including BasestationConfig
def read_basestation_config(config_name = 'valence_pod'):
    return BasestationConfig(config.read(config_name))

in notebook.ipynb I have a test cell:

import lib.basestation_config as bsc
bs_config = bsc.read_basestation_config()
display(bs_config)

and when I run it, I get:

<module 'config' (namespace)>
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-f2942fb5fb2d> in <module>
      1 import lib.basestation_config as bsc
----> 2 bs_config = bsc.read_basestation_config()
      3 display(bs_config)

/mnt/eng_store/pod/logs/embedded/utils/logutils/indot-py/lib/basestation_config.py in read_basestation_config(config_name)
    270 def read_basestation_config(config_name = 'valence_pod'):
    271     print(config)
--> 272     return BasestationConfig(config.read(config_name))

AttributeError: module 'config' has no attribute ‘read’

Answer 1

When you import config, Python is using the config folder (the one with the JSON file) instead of the lib/config.py module. You can check this by printing config.__path__ after importing it:

import config as config
print(config.__path__)
# _NamespacePath(['/path/to/root/config']) 

_NamespacePath indicates that the folder config is being treated as an implicit Python package, it does not contain an __init__.py like a regular Python package, but the name matches the "config" being imported.

While looking for a module or package named "foo", for each directory in the parent path:

• If <directory>/foo/__init__.py is found, a regular package is imported and returned.
• If not, but <directory>/foo.{py,pyc,so,pyd} is found, a module is imported and returned. The exact list of extension varies by platform and whether the -O flag is specified. The list here is representative.
• If not, but <directory>/foo is found and is a directory, it is recorded and the scan continues with the next directory in the parent path.
• Otherwise the scan continues with the next directory in the parent path.

Your setup falls under bullet #3, as <directory>/config matches the import config target. You might then wonder what is <directory> here? That depends on the Module Search Path, stored in sys.path, which is a list of all the directories where Python will look for import targets. When you run your test script under root, the root directory is then added to your sys.path.

The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path.

Check it by adding this before import config in basestation_config.py:

import sys
print(sys.path)
# ['/path/to/root', ... ]

import config as config

That explains the why. To fix this, you can do this:

1. Rename the config folder to something else (ex. jsonfiles), just to prevent future errors like this and to distinguish it from the config.py module

2. Change lib to follow the regular Python package structure, by adding a __init__.py file under lib to clearly mark it as a package.

   lib
   ├── __init__.py
   ├── basestation_config.py
   └── config.py
   

3. Lastly, make it clear in basestation_config.py that you are importing the config.py in the same directory.

   from . import config as config    
   print(config.__file__) 
   # /path/to/lib/config.py
   

Note that, before doing step 3, if you added the print(config.__path__) earlier, make sure to remove it after applying the corrected codes, since it's mostly likely not available on your config.py (you might get "AttributeError: module 'lib.config' has no attribute __path__").

It should now work after those changes.