how to add a package to sys path for testing

Question

This question is occasioned by instructions in the python guide for adding a project to sys path to use in tests, which do not seem to work unless I am misunderstanding the instructions

I have a directory structure for a python project like this

sample/a.py
sample/b.py
sample/c.py
sample/__init__.py
test/context.py
test/test_something.py
test/__init__.py
docs

According to the python guide, I should create a test/context.py file and add this

import os
import sys
sys.path.insert(0, os.path.abspath('..'))

import sample

Then, in my test/test_something.py file, it says I can do this

from .context import sample

The guide says "This will always work as expected".

but, when I cd into test and run

python -m unittest test_something

I get an error

ValueError: Attempted relative import in non-package

and the error message specifically refers to this: from .context import sample

Question: How can I add my sample package to the sys path correctly?

When answering, can you also clarify if the solution will handle absolute imports within the sample package. For example, my sample.a imports sample.b etc. When I had my tests structured a different way, I did an absolute import of sample.a, but since it has a relative import of from .b import Boo, it produced a similar error

Update

`File "/usr/local/lib/python2.7/runpy.py", line 162 in _run_module_as_main "__main__", fname, loader, pkg_name)
File  "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals
File  "/usr/local/lib/python2.7/unittest/__main__.py", line 12, in module main(module=None)
File  "/usr/local/lib/python2.7/unittest/main.py", line 94, in __init__ self.parseArgs(argv)
File  "/usr/local/lib/python2.7/unittest/main.py", line 149 in parseArgs self.createTests()
File  "/usr/local/lib/python2.7/unittest/main.py", line 158, in createTests self.module)
File  "/usr/local/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames suites = [self.loadTestsFromName(name,module) for name in names]
File  "/usr/local/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = __import__('-'.join(parts_copy))
File "test_something.py", line 8, in module from .context import sample

Update

if I run the following command from root directory

   python -m unittest test

It says , "Ran 0 tests in 0.000s"

If, as was suggested in the comments by @cuongnv, I run this from root directory

python -m unittest test/test_something.py

or this (without the file extension)

python -m unittest test/test_something 

It says "Import by filename is not supported"

Answer 1

I had a battle to get my testing directory structure to work outside of an IDE. Please find my solution below. Tested on Windows 7 using python 3.6 and Linux Mint using python 3.4, running the code using the command line:

python -m pytest test_compress_files.py

The file I wrote to be tested is called compress_files.py in a directory named \src. The file containing tests to be run using pytest is called test_compress_files.py in a subdirectory \tests, so the full directory path is \src\tests. I needed to add a file called context.py to the \src\tests directory. This file is used in test_compress_files.py to enable access to compress_files.py in the directory above. The _init_.py files are empty.

Directory structure

\src
__init__.py
compress_files.py

\src\tests
__init__.py
context.py
test_compress_files.py  

compress_files.py contains the script to be tested.

context.py:

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import compress_files  

The line:

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

comes from the suggestion at the hitch hikers guide to python at http://docs.python-guide.org/en/latest/writing/structure/. This adds the path of the directory above the /src/tests directory to sys.path, which in this case is /src.

test_compress_files.py:

import os
import pytest
from context import compress_files
from compress_files import *

# tests start here
...

Answer 2

This is a bit old, and slightly tangential, but this question/answers helped me when trying to patch on pandas.

I had imported pandas the "standard" way as pd:

import pandas as pd

So when declaring the import path I had to do:

unittest.mock.patch("src.path.to.file.pd")

NOT

unittest.mock.patch("src.path.to.file.pandas")

Answer 3

I can't comment yet but i was trying out Oppy's answers in python 3.7 env and it fail as:

ImportError: cannot import name 'compress_files' from '__main__' (test_compress_files.py)

I resolved using a modified test_compress_files.py (notice the removed dot) :

import os
import pytest
from context import compress_files
from compress_files import *

# tests start here
...

Answer 4

Question: How can I add my sample package to the sys path correctly?

You're doing it the right way, but you missed declaring your folder to be a package. Try solution of Christian, it should work.

Your path is stored in sys.path. By doing this:

sys.path.insert(0, os.path.abspath('..'))

You're telling your python to add upper folder (of current file) into your path. As sys.path is a list, you can using other methods of list like insert, append...

In your case, you're inserting your upper dir at top of the path list.

See:

In [1]: import sys

In [2]: sys.path
Out[2]: 
['',
 '/usr/local/bin',
 '/usr/lib/python3.4',
 '/usr/lib/python3.4/plat-x86_64-linux-gnu',
 '/usr/lib/python3.4/lib-dynload',
 '/usr/local/lib/python3.4/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/usr/local/lib/python3.4/dist-packages/IPython/extensions',
 '/home/cuong/.ipython']

In [3]: sys.path.insert(0, '/tmp/foo')

In [4]: sys.path
Out[4]: 
['/tmp/foo', **<-- on top**
 '',
 '/usr/local/bin',
 '/usr/lib/python3.4',
 '/usr/lib/python3.4/plat-x86_64-linux-gnu',
 '/usr/lib/python3.4/lib-dynload',
 '/usr/local/lib/python3.4/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/usr/local/lib/python3.4/dist-packages/IPython/extensions',
 '/home/cuong/.ipython']

So, from here, when you have

import sample

your python will try to look in path to see if there is any sample package.

Unfortunately, it can't find sample as you didn't make it as a package because your forgot __init__.py in sample folder.

Hope my explanation would help you to understand and you can handle other situations different to this.

Answer 5

Try adding an empty __init__.py to tests/: touch tests/__init__.py should do it.