CODEWITHSUNDEEP
maventestingrelease
leif.gruenwoldt
leif.gruenwoldt

Reputation: 13967

Testing SNAPSHOT's vs. release artifacts

We use what I think is a pretty standard maven-release-plugin style of development. The master branch contains development toward our next release (pom labeled x.y-SNAPSHOT). When we go into code freeze we branch from master to prepare the release. We perform releases from this branch, any bugs are fixed on this this branch.

Now for my question. When readying for a x.y release we have typically done testing against a x.y-SNAPSHOT, built from this release branch. However we realized that when testing "passed", it had passed against an installer with a SNAPSHOT label. So to perform a release we would have to change the code (drop the SNAPSHOT label) and re-spin a new release. In our opinion, re-spinning a build just invalidated any testing we had done against the SNAPSHOT -- requiring us to re-test the final release.

What to do?

I'm considering recommending we only perform our formal tests against non SNAPSHOT builds. If bugs are found, in what are essentially "release candidates", we fix them in the release branch and bump the version x.y.(z+1), and re-test. The downside is that instead of a clean x.y.0 release it's now named x.y.z, where z is the number of release candidates this took.

Anyone have experience with a scenario like this? Is this a normal process or are we over-reacting about testing SNAPSHOT's?

Upvotes: 4

Views: 401

Answers (1)

Stephen Connolly
Stephen Connolly

Reputation: 14096

There are generally two ways this is handled:

  1. The maven project itself uses the Staging feature of some repository managers to hold the artifacts in a staging area where they can be tested. If the tests fail, the staging repo is dropped, the tag deleted, and the release respun

  2. In other situations the view is that version numbers are cheap (sure there's an infinite supply), so if the release is borked, just respin the next number

The first means you don't need to track which releases are good/bad, but requires people to verify that the correct artifacts are being tested.

The second requires some additional tooling to identify the status of each version and doesn't require modifying tags.

Either works, pick your poison ;-)

Upvotes: 2

Related Questions