CODEWITHSUNDEEP
unit-testingcontinuous-integration
Corpus Gigantus
Corpus Gigantus

Reputation: 595

What kind of tests should run on the CI server?

Currently, our unit tests are committed along with the application code and executed by a build bot job on each commit. Likewise, code coverage is calculated. However, both UTs and coverage are - or can be - conducted by the developers before they commit new features to the repository, so the CI process does not seem to add any value.

What kind of tests do you suggest should be executed on the CI server which are not executed by the developer's prior to commit?

Upvotes: 2

Views: 599

Answers (3)

ekostadinov
ekostadinov

Reputation: 6940

Completely agree with previous posts, as Unit testing is mostly done by devs. So tests that should be executed as part of the CI process is pretty much opinion based. Depending of the goals of the team/project.

The thing that is also important is that CI (server) gives you a separate testing environment. So your test effort and execution can run independent. Your test are executed in a clone of the production environment.

In my expirience I've used CI server mostly for System,Integration, Functional, Regression testing and UAT.

Upvotes: 1

Floegipoky
Floegipoky

Reputation: 3273

David raises very good points. One thing that he didn't mention though is that automated testing in a continuously integrated environment can be even more powerful when it goes beyond unit tests. The CI process allows you to run integration and system level tests that would be too expensive to run on a dev box. For example, you may have unit tests for your persistence layer that run against an in-memory database. Your CI server however can run these same automated tests against a snapshot of your production database.

Upvotes: 1

David
David

Reputation: 218837

so the CI process does not seem to add any value

No?

  • What happens if a developer doesn't run the tests before committing?
  • What happens if a developer commits a change which conflicts with a change committed by another developer?
  • What happens if the tests pass on the developer's machine but not on other machines?
  • What happens if another developer has changed a test?
  • ...

CI isn't "continuous testing", it's "continuous integration". The need to run the tests as part of the integration build is to validate that the changes committed can successfully integrate with what's already there. Whether or not the tests passed on the developer's local potentially-non-integrated workstation is immaterial.

The unit tests (any reasonably fast automated tests, really) should be executed by the CI server to validate the current state of the build. That state may be different than what is on one individual developer's workstation.

They may be the same tests which were just run by the developer moments prior. But the context in which the tests run is physically different and the need to run them is semantically different. Unless you're being charged by the clock cycle, there's no reason to omit running tests.

Upvotes: 3

Related Questions