CODEWITHSUNDEEP
testinglanguage-agnosticrefactoringintegration-testingregression-testing
Seth Reno
Seth Reno

Reputation: 5450

Is it bad practice to remove and combine regression tests to speed up testing?

I'm working on an app that integrates with a 3rd party web service. I currently have separate integration / regression tests that call the web service to do the following:

Most of these tests were created as bugs were found & fixed. The 3rd party web service is slooow and I'm trying to speed the testing process up. Because each test calls the web service, combining them into one test that only calls the web service once would make things much faster.

Would combining these tests be bad practice because each test was written for a specific bug? My concern is that a mistake in refactoring could potentially allow a bug to be re-introduced later on.

Upvotes: 2

Views: 196

Answers (3)

artbristol
artbristol

Reputation: 32407

I wouldn't recommend combining them, unless you keep the ability to run them separately (maybe keep them separate in your overnight build, and combined in your continuous build).

Try parallelizing them (on separate 'policies'), if your test framework supports it.

Upvotes: 1

Buh Buh
Buh Buh

Reputation: 7546

I would suggest including them in your nightly build, so that they do run once a day when you are asleep and not watching the clock. And only removing them your developer time tests.

Of course that assumes they are not soooo sloooow that one night is not enough.

Just combining your tests into one big test is likely to make them useless or worse. Thats's not much better than just deleting them.

Upvotes: 0

Carl Manaster
Carl Manaster

Reputation: 40336

Yes, combining them would be a bad practice. Think instead about how to mitigate the risk without combining the tests. One approach - probably your best bet - would be to mock out the web service, so that the tests are much faster without jeopardizing their ability to detect a regression. Another would be to split your slow regression tests into their own suite that is run less frequently (but still frequently enough!) than your usual set of tests. Finally, you could combine them - but I would recommend explicitly reintroducing all the original bugs into your code to verify that the combined test still detects them.

Specific, pointed, direct, unit tests are very valuable; it's nice to know exactly what has broken. Combining tests compromises that value.

Upvotes: 1

Related Questions