Within the py.test test directory for a python module, import a file for the module

Question

I have a python module arranged in a directory. Call it foo.

Here is the file layout:

caller.py
foo/__init__.py
foo/bar.py
foo/test/bar_test.py

That is, the module is called foo and the code in foo/__init__.py gets imported when the import foo statement in caller.py gets run.

Within foo/__init__.py, I wish to access the contents of bar.py. This is done with import foo.bar.

My problem arises when writing the code that runs in foo/test/bar_test.py. If this file was simply foo/bar_test.py, then it could also use import foo.bar to import the contents of foo.bar.py. Unfortunately, we have a coding standard that says that unit tests go in a subdirectory called tests.

Given that coding standard, how do we import bar.py?

This doesn't work:

# foo/test/bar_test.py
import foo.bar
def test_return5():
   assert bar.return5() == 5

It gives me:

$ py.test
================================================== test session starts ==================================================
platform linux -- Python 3.6.3, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/hadoop/xxx/foo, inifile:
collected 0 items / 1 errors

======================================================== ERRORS =========================================================
___________________________________________ ERROR collecting test/bar_test.py ___________________________________________
ImportError while importing test module '/home/hadoop/xxx/foo/test/bar_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test/bar_test.py:3: in <module>
    import foo.bar
E   ModuleNotFoundError: No module named 'foo'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================ 1 error in 0.10 seconds ================================================

This works, but it's gross:

# foo/test/bar_test.py

import os
import sys

sys.path.append( os.path.join(os.path.dirname(__file__), "../.."))

import foo.bar

def test_return5():
   assert foo.bar.return5() == 5

This does not work:

# foo/test/bar_test.py

import os
import sys

sys.path.append( os.path.join(os.path.dirname(__file__), ".."))

import bar

def test_return5():
   assert bar.return5() == 5

Because:

$ py.test
================================================== test session starts ==================================================
platform linux -- Python 3.6.3, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/hadoop/xxx/foo, inifile:
collected 0 items / 1 errors

======================================================== ERRORS =========================================================
___________________________________________ ERROR collecting test/bar_test.py ___________________________________________
ImportError while importing test module '/home/hadoop/xxx/foo/test/bar_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test/bar_test.py:8: in <module>
    import bar
bar.py:3: in <module>
    import foo
E   ModuleNotFoundError: No module named 'foo'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================ 1 error in 0.10 seconds ================================================

Because bar.py has:

# foo/bar.py

import foo

def return5():
    return 5

Answer 1

import foo.bar as bar

That should work in bar_test.py if you run pytest from the parent directory (at least, it worked for me just now).

You can't use import bar in python 3 (I'm assuming that's what you're using) because it's ambiguous - see PEP-404 and a more in depth stackoverflow answer for more details.

EDIT: Looking at the expanded example, it looks like the problem is that bar_test.py isn't part of the foo package (because the test directory isn't a package). If you add a __init__.py file to the test directory, pytest will happily run the test::

$ pytest foo/test/bar_test.py 
===================================================================== test session starts ======================================================================
platform linux -- Python 3.5.3, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /home/xxxx/tmp/blank, inifile:
collected 1 item                                                                                                                                                

foo/test/bar_test.py .                                                                                                                                   [100%]

=================================================================== 1 passed in 0.03 seconds ===================================================================
$ ls -lh foo/test/
total 12K
-rw-rw-r-- 1 xxxx xxxx   95 Jul 26 14:44 bar_test.py
-rw-rw-r-- 1 xxxx xxxx    0 Jul 27 16:38 __init__.py
-rw-rw-r-- 1 xxxx xxxx  132 Jul 27 16:38 __init__.pyc
drwxrwsr-x 2 xxxx xxxx 4.0K Jul 27 16:38 __pycache__

Note that I had to delete the __pycache__ after adding __init__.py.

See the pytest docs for more information.