CODEWITHSUNDEEP
pythontestingpytestpython-unittest
Snusifer
Snusifer

Reputation: 553

Unittest's subTest doesn't work with Pytest, or am I crazy?

As I understand, pytest should provide parameter information withunittest.TestCase.subtest() if a test fails. So here is something that resembles my code:

class TestStuff(unittest.TestCase):

    def test_foo(self):
        for i in range(0, 100):
            with self.subTest(msg = "seed", i = i):
                np.random.seed(i)
                n = np.random.randint(0, 30)
                self.assertGreaterEqual(28, n)

This ofcourse fails, and prints out the following:

================================================================================================== test session starts ===================================================================================================
platform darwin -- Python 3.7.3, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /Users/foopackage
plugins: openfiles-0.3.2, arraydiff-0.3, doctestplus-0.3.0, remotedata-0.3.1
collected 7 items                                                                                                                                                                                                        

foo.py ......F                                                                                                                                                                                               [100%]

======================================================================================================== FAILURES ========================================================================================================
________________________________________________________________________________________________ TestStuff.test_foo _________________________________________________________________________________________________

self = <foo.test_foo.TestStuff testMethod=test_foo>

    def test_foo(self):
        for i in range(0, 100):
            with self.subTest(msg = "seed", i = i):
                np.random.seed(i)
                n = np.random.randint(0, 30)
>               self.assertGreaterEqual(28, n)
E               AssertionError: 28 not greater than or equal to 29

foo.py:135: AssertionError
=========================================================================================== 1 failed, 6 passed in 1.91 seconds ===========================================================================================

As you can see, there are no message about which seed (value of i) failed the test. I have read everywhere that Pytest is compatible with unittest, so I cant seem to see the problem here. Can someone explain this? Thx

Upvotes: 4

Views: 2059

Answers (3)

Alexander Duffaut
Alexander Duffaut

Reputation: 11

Simply installing pytest-subtest worked for me: pip install pytest-subtest

Upvotes: 1

Alexander Troshchenko
Alexander Troshchenko

Reputation: 330

For people coming in like me, check if you are using Nosetests instead. I've added it to our project at some point to get test coverage out and several months later we noticed that subTest don't execute separately (when stuff broke, as usual), and were trying to figure out why all of the sudden subTests are executing inline with main tests.

Other things to look that are not directly related but might help:

  1. If you are using PyCharm, pay attention to how your tests are running. It might explicitly run them with nosetest, pytest or unittest instead of Django manage.py RunTests.
  2. In PyCharm you (or someone else) can switch off Django testcase runner, you (see PyCharm > Settings > Languages & Frameworks > Django > Do not use Django test runner)
  3. In PyCharm you can also select a default testcase runner forcibly (for new configs). See PyCharm > Settings > Tools >Python Integrated Tools > Testing (section) > Default test runner
  4. In settings.py of your Django app you can also specify a test runner. Mine was, again, nosetest, which doesn't properly work as far as PyCharm & Django goes (for me).
  5. Make sure your test runner IS compatible with subTests. As far as I know pytest and unittest are fine. Everything else your mileage may vary
  6. Make sure you are inheriting from a correct TestCase. django.test.TestCase is unittest.TestCase and supports subTest since v3.4 (of unittest)

Upvotes: 0

user78110
user78110

Reputation:

the pytest-subtest plugin is needed to add that feature

Upvotes: 4

Related Questions