AttributeError: 'ImportDenier' object has no attribute 'find_spec' when loading function

Question

Description

I have a module exactly as follows

my_module
   |-> _sub_module
   |     |-> test_fun.py
   |-> test_fun1.py
   |-> run.py
   |-> __init__.py

the content of the files is the followng

test_fun.py

def test_fun():
    pass

test_fun1.py

def test_fun1():
    pass

run.py

from my_module.test_fun1 import test_fun1
from my_module._sub_module.test_fun import test_fun

while __init__.py is empty.

Problem

when I run run.py (from the module parent directory) it works just fine. However, if I comment the first line in run.py and execute only the second one (from my_module._sub_module.test_fun import test_fun) I get the following error

runfile('C:/Users/###/Documents/GitLab/test_project/my_module/run.py')
Reloaded modules: my_module, my_module.test_fun1, my_module._sub_module.test_fun
Traceback (most recent call last):

  File "<frozen importlib._bootstrap>", line 908, in _find_spec

AttributeError: 'ImportDenier' object has no attribute 'find_spec'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\###\Documents\GitLab\test_project\my_module\run.py", line 4, in <module>
    from my_module._sub_module.test_fun import test_fun

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 910, in _find_spec

  File "<frozen importlib._bootstrap>", line 884, in _find_spec_legacy

  File "C:\Users\###\Anaconda3\lib\site-packages\IPython\external\qt_loaders.py", line 48, in find_module
    if path:

  File "<frozen importlib._bootstrap_external>", line 1183, in __len__

  File "<frozen importlib._bootstrap_external>", line 1162, in _recalculate

  File "<frozen importlib._bootstrap_external>", line 1158, in _get_parent_path

KeyError: 'my_module'

Why is this happening? How can the first import "fix" the second one since they import completely unrelated things (apart from a similar name) ?

I'm running Python 3.8.5 from Spyder installed inside Anaconda

EDIT 1

As @Dima Berehovets correctly pointed out, when running run.py I cannot do it from inside \my_module as python does not check the name of the parent directory, but rather looks for a folder my_module inside my_module.

In my previous tests I was indeed running the code from the parent directory of my_module.

To make the example even more trivial, I edited the structure of the project as @Dima Berehovets suggested, moving run.py up one level.

The behavior now, however, is even weirder!

Now, when I run the code the first time it runs fine (always with only the second line in run.py). But if I run it a second time immediately after, the error pops up as before.

runfile('C:/Users/###/Documents/GitLab/test_project/run.py', wdir='C:/Users/###/Documents/GitLab/test_project')

runfile('C:/Users/###/Documents/GitLab/test_project/run.py', wdir='C:/Users/###/Documents/GitLab/test_project')
Reloaded modules: my_module, my_module._sub_module.test_fun
Traceback (most recent call last):

  File "<frozen importlib._bootstrap>", line 908, in _find_spec

AttributeError: 'ImportDenier' object has no attribute 'find_spec'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\###\Documents\GitLab\test_project\run.py", line 4, in <module>
    from my_module._sub_module.test_fun import test_fun

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 910, in _find_spec

  File "<frozen importlib._bootstrap>", line 884, in _find_spec_legacy

  File "C:\Users\###\Anaconda3\lib\site-packages\IPython\external\qt_loaders.py", line 48, in find_module
    if path:

  File "<frozen importlib._bootstrap_external>", line 1183, in __len__

  File "<frozen importlib._bootstrap_external>", line 1162, in _recalculate

  File "<frozen importlib._bootstrap_external>", line 1158, in _get_parent_path

KeyError: 'my_module'

EDIT 2.1

I can confirm that I cannot replicate the error executing the code from a console. I'm starting to think that this might be Spyder-related

sub-edit: Also in an IPython console this seems to work

I added the Spyder tag to the question

Answer 1

When you run run.py, the program considers the folder 'my_module' as the root folder of the project. Usually, in python programs, there should be a .py file in the project root wich will be an entry point to the program. For example, let's say you have the following file structure:

project:
    module1:
       -views.py
    module2:
       -views.py
    run.py

In this case, you would start your program using the file run.py and you would do some imports in this file using the structure

from module1.views import some_function

However, in your case, when you run run.py from the folder my_module and make imports like this

from my_module._sub_module.test_fun import test_fun

The program tries to find a folder my_module in your folder my_module. If you really need to run the file run.py from my_module, make imports without adding 'my_module' like shown below

from test_fun1 import test_fun1
from _sub_module.test_fun import test_fun

However, I would recommend you to keep the structure of the project, as I've shown above. Hope, this will help.