Run all tests from subdirectories in Python

Question

I am at my wits end with trying to get all my unittest to run in Python. I have searched about 30 different posts and the unit test documentation but still cannot figure it out.

First I have two test classes that I can run each individually and all the tests pass:

File: unittest.subfolder1.TestObject1.py

class TestObject1(unittest.TestCase):
  def test_case1(self):
    ...some code...
    ...some assertions...

if __name__ == '__main__':
  unittest.main()

File: unittest.subfolder2.TestObject2.py

class TestObject2(unittest.TestCase):
  def test_case1(self):
    ...some code...
    ...some assertions...

if __name__ == '__main__':
  unittest.main()

Starting in the top level directory above 'unittest' I am trying to us unittest.discover to find and run all my tests:

import unittest

loader = unittest.TestLoader()
suite = loader.discover('unittest')
unittest.TextTestRunner().run(suite)

When I do this I get the error `ModuleNotFoundError: No module named 'subfolder1.TestObject1'

What am I doing wrong?

Answer 1

Testing All Subdirectories

Given a structure of:

my_package
|
|
controller
|-- validator.py
|
validator
|-- controller.py
|
test
|-- controller
|
  |-- __init__.py (MAKE SURE THIS EXISTS OR unittest MODULE WOULD NOT KNOW)
  |-- test_controller.py
  |
|-- validator
|
  |-- __init__.py (MAKE SURE THIS EXISTS OR unittest MODULE WOULD NOT KNOW)
  |-- test_validator.py
  |

then just run

python -m unittest discover -s my_package/test

What this does is to test and -s means to start with the my_package/test as the starting directory

Answer 2

In my project all folders are folders (not modules) and they have the structure:

Folder > Subfolder > Subfolder > Tests > test_xxxx.py
Folder > Subfolder > Subfolder > xxxx.py

So i modified the answer from here, and also took a part from How do I run all Python unit tests in a directory? and came up with this:

import os, unittest

testFolderPaths = []

for path, subdirs, files in os.walk(os.getcwd()):
    for file in files:
        if file.startswith("test_") and file.endswith(".py"):
            testFolderPaths.append(path)

for path in testFolderPaths:
    print(f"Running tests from {path}...")
    loader = unittest.TestLoader()
    suite = loader.discover(path)
    runner = unittest.TextTestRunner()
    result = runner.run(suite)
    print(f"RUN {result.testsRun} Tests. PASSED? {result.wasSuccessful()}")

If any tests fail it will throw and error showing which one exactly failed.

Answer 3

Old question but, oh, so current. I am new to Python, coming from strong typed languages and while the language itself is ok(ish), the conventions, tools and workarounds to make everything work in the ecosystem can drive you nuts. I struggled myself with running unit tests from separate subdirectories and this is the way I solved it. First, the code you test, package it into a package. Organize your directories like this:

Work
|
+---PkToTest
|   |
|   +--- __init__.py
|   +--- a.py
|   +--- <other modules>.py
|
+---Tests (for PKToTest)
    |
    +--- test_a.py

PkToTest becomes a package due to the init.py file. In test_a.py make sure your sys.path will contain the path to PkToTest (absolute path not relative). I did that by:

import sys
sys.path.insert(0, "<absolute path to parent of PkTotest directory>")

import unittest
from PkToTest import a

class aTestSuite(unittest.TestCase):
    def test1(self):
       self.assertEqual(a.fnToTest(), ...)

Answer 4

A good approach is to run all the tests in a subdirectory from the command line. In order to find the following files "TestObject1.py, TestObject2.py, ..." in subdirectories, you can run the following command in the command line:

python -m unittest discover -p 'Test*.py'

Additionally, the __init__.py is required within the import and module directories: Python unittest discovery with subfolders

The import unittest is required in the files unittest.subfolder1.TestObject1.py and unittest.subfolder2.TestObject2.py

It is also possible to define explicitly the directory where the discovery starts with the -s parameter:

python -m unittest discover [options]

-s directory     Directory to start discovery ('.' default)
-p pattern       Pattern to match test files ('test*.py' default)

In case you are using unittest2, it comes with a script unit2. The command line usage is:

unit2 discover unit2 -v test_module

Answer 5

So I had to do my own workaround but at least I can get them all to run with the above file structure. It requires that I reinstantiate the TestLoader and the TestSuite each time I give it a new file path, so first I need to collect all relevant file paths in the unittest directory.

import os
import unittest
import traceback


class UnitTestLauncher(object):

  def runTests(self):
    #logging.INFO("Running unit tests...")


    lsPaths = []

    #Find all relevant subdirectories that contain unit tests
    #Exclude 'unittest' directory, but include subdirectories, with code `path != 'unittest'`
    for path,subdirs,files in os.walk('unittest'):
      if "pycache" not in path and path != 'unittest':
        lsPaths.append(path)

    #loop through subdirectories and run individually
    for path in lsPaths:
      loader = unittest.TestLoader()
      suite = unittest.TestSuite()
      suite = loader.discover(path)
      unittest.TextTestRunner().run(suite)

This solution is not perfect and each different directory comes out as a line of output so you have to look through each line manually for failed tests.

Answer 6

Do not name your directory unittest, it may conflict with the standard library.

You also need to create a file named __init__.py in all of your directories (subfolder1, etc.), so they become packages and their content can be imported.