CODEWITHSUNDEEP
subliminal
Ray
Ray

Reputation: 501

How do I get subliminal to continue with test suite after 1 test in suite fails?

Currently my app has a somewhat unstable development environment and a few of my tests will result in a false negative. If one of my tests fails in my Subliminal test suite, the entire suite stops! This is not what I want and I dont want to do something like surrounding my tests in try/catch blocks so how do I prevent my Subliminal test suite from stopping after one fail?

Upvotes: 1

Views: 102

Answers (1)

Aaron Golden
Aaron Golden

Reputation: 7102

There are two scenarios in which an exception thrown as part of a Subliminal test suite will cause the whole suite to stop. These are:

  1. If the exception occurs inside of the setUpTest method, which runs before the test cases in the test suite begin executing. In this case none of the test cases will execute.
  2. If the exception is "unexpected" meaning that it occurs outside of any SLAssert macro. Expected exceptions are the ones that the test case (or setUpTest for that matter) actually look for via SLAssert calls. Any other exceptions are considered to be essentially bugs in the test suite/case itself---something went wrong that the tests weren't prepared to handle. So the suite stops at that point instead of continuing with the app in a state it might not be able to handle.

From your comment it sounds like you ran into the second case. You could probably get the tests to continue by adding additional SLAssert calls to look for the unexpected issue that you're seeing (or by resolving whatever issue caused the exception in the first place).

Upvotes: 1

Related Questions