CODEWITHSUNDEEP
c#unit-testingregression-testing
Madhur Maurya
Madhur Maurya

Reputation: 1070

Test Method Fails in Test Suite but passes individually in .net C#

I am encountering a very strange problem in .net regression testing. I have a test method which fails when I run the complete test suit, but the same test method passes when run individually.

What could be the possible reason behind it. I double checked that other test methods are having no effect on it. The test method uses web service to fetch some data, I do not have any idea why it gets incorrect data when run in the suite.

Has anyone faced the similar situation earlier.

Upvotes: 1

Views: 1899

Answers (3)

CubanX
CubanX

Reputation: 5252

You most likely have test pollution somehow. Check any global state you create to make sure it's reset to the desired state before running a test.

Use copious amounts of logging to get to the source of it :)

Upvotes: 1

Robert Snyder
Robert Snyder

Reputation: 2409

I have also encountered the same problem and it isn't just exclusive to the Microsoft framework. I've seen this in NUnit as well. For me (and most commonly) it is that you are accessing the same item in multiple tests so that the state of it is not as you would expect when run as a group. It's kind of hard to spot at first but if you have in your mind when writing your tests that any test should be able to run at anytime in almost any circumstances you should be good to go.

To help hunt down what the problem is try Asserting things about the state that you assumed was true (things such as Id == 1, or Name == null, or IsConnected == false) if you can get it to fail before you even start that you can be sure it is somethign to do with state.

Upvotes: 0

atlaste
atlaste

Reputation: 31106

I am encountering a very strange problem in .net regression testing. I have a test method which fails when I run the complete test suit, but the same test method passes when run individually.

I've encountered similar things before, usually having to do with:

  1. Application settings. This might be the case with a web service. Note that your tests run in a test application, not in your normal 'exe' application.
  2. Static variables / initialization. If you have a [testmethod] and a [testclass], the class will be instantiated once, after which all methods will be called. If you use state in your class, things can get broken.

In all cases, easiest way to figure out what's going on is probably to put a breakpoint before calling the service.

Upvotes: 2

Related Questions