CODEWITHSUNDEEP
pythonunit-testing
AnC
AnC

Reputation: 4201

PyUnit: stop after first failing test?

I'm using the following code in my testing framework:

testModules = ["test_foo", "test_bar"]

suite = unittest.TestLoader().loadTestsFromNames(testModules)
runner = unittest.TextTestRunner(sys.stdout, verbosity=2)
results = runner.run(suite)
return results.wasSuccessful()

Is there a way to make the reporting (runner.run?) abort after the first failure to prevent excessive verbosity?

Upvotes: 26

Views: 10533

Answers (4)

moink
moink

Reputation: 888

Nine years after the question was asked, this is still one of the top search results for "python unit test fail early" and, as I discovered when looking at the other search results, these answers are no longer correct for more recent versions of the unittest module.

The documentation for the unittest module https://docs.python.org/3/library/unittest.html#command-line-options and https://docs.python.org/2.7/library/unittest.html#command-line-options show that there is an argument, failfast=True, that can be added to unittest.main, or equivalently a command line option, -f, or --failfast, to stop the test run on the first error or failure. This option was added in version 2.7. Using that option is a lot easier than the previously-necessary workarounds suggested in the other answers.

That is, simply change your

unittest.main()

to

unittest.main(failfast=True)

Upvotes: 36

philwhln
philwhln

Reputation: 395

Building on AnC's answer, this is what I'm using...

def aborting_run(self, result=None):
    if result.failures or result.errors:
        print "aborted"
    else:
        original_run(self, result)
original_run = unittest.TestCase.run
unittest.TestCase.run = aborting_run

Upvotes: 0

AnC
AnC

Reputation: 4201

Based on Eugene's guidance, I've come up with the following:

class TestCase(unittest.TestCase):  
    def run(self, result=None):
        if result.failures or result.errors:
            print "aborted"
        else:
            super(TestCase, self).run(result)

While this works fairly well, it's a bit annoying that each individual test module has to define whether it wants to use this custom class or the default one (a command-line switch, similar to py.test's --exitfirst, would be ideal)...

Upvotes: 7

Eugene Morozov
Eugene Morozov

Reputation: 15816

It's a feature. If you want to override this, you'll need to subclass TestCase and/or TestSuite classes and override logic in the run() method.

P.S.: I think you have to subclass unittest.TestCase and override method run() in your class:

def run(self, result=None):
    if result is None: result = self.defaultTestResult()
    result.startTest(self)
    testMethod = getattr(self, self._testMethodName)
    try:
        try:
            self.setUp()
        except KeyboardInterrupt:
            raise
        except:
            result.addError(self, self._exc_info())
            return

        ok = False
        try:
            testMethod()
            ok = True
        except self.failureException:
            result.addFailure(self, self._exc_info())
            result.stop()
        except KeyboardInterrupt:
            raise
        except:
            result.addError(self, self._exc_info())
            result.stop()

        try:
            self.tearDown()
        except KeyboardInterrupt:
            raise
        except:
            result.addError(self, self._exc_info())
            ok = False
        if ok: result.addSuccess(self)
    finally:
        result.stopTest(self)

(I've added two result.stop() calls to the default run definition).

Then you'll have to modify all your testcases to make them subclasses of this new class, instead of unittest.TestCase.

WARNING: I didn't test this code. :)

Upvotes: 7

Related Questions