CODEWITHSUNDEEP
unit-testingcontinuous-integration
NBPC77
NBPC77

Reputation: 277

Continuous Integration - unit tests only? Or both unit and integration tests?

Having an internal debate as we move into CI.

One side argues that only unit tests should be included on the CI server whereas the other side argues that both unit and integration tests should be run on the server.

Just to be clear: I consider unit tests to be tests that have NO external/3rd party system dependencies (i.e., DB, File systems, web services, etc) and integration tests are those that do.

Thank you for any guidance. If you could provide references to your side of the argument, that would be appreciated.

Thanks, NB

Upvotes: 1

Views: 228

Answers (1)

lockstock
lockstock

Reputation: 2427

The general idea would definitely be that integration tests should NOT be included.

However, that's a general rule and may not be true in your case. If your integration tests are stable enough that the benefit from running them regularly out weighs the disadvantages of having to maintain them, then you should include them.

I consider integration tests to be tests that rely on anything that is not included in the build output, which sounds like basically the same as your definition.

Its also not a black and white issue. There are different degrees of stability. For example in decreasing order:

  • unit tests
  • integration tests that build an external db, test, and then destruct the db
  • integration tests that rely on static data in a db that exists purely for the tests
  • integration tests that rely on a development db that is used for other purposes
  • integration tests that rely on a third party web service

Some people may suggest that with the correct use of dependency injection and mocking frameworks that there should never be a need for integration tests. I'll leave that for your team to debate.

There are also different degrees of consequences of including tests. For example failing the build upon failure of tests, or simply sending notifications and/or requiring some sort of approval point.

Here's a possible approach:

  • include all unit tests
  • decide which integration tests would be of benefit to include (low maintenance and within your control)
  • have a way of quickly removing the integration tests from the build definition in the case that something goes wrong that causes the tests to fail, but that you don't want to result in build failure (dev db goes down for example)
    • you can do this by having multiple build definitions that you can switch between
  • decide on the appropriate action for failed tests (build failure, notification, company wide email with photo of dev wearing a dunces hat etc..)

hope that helps!

Upvotes: 0

Related Questions