CODEWITHSUNDEEP
mavenjenkinshudsonbuild-processgradle
liam.j.bennett
liam.j.bennett

Reputation: 424

Setting up a Staging process for Java projects

I currently have a large number of projects with a complex dependency structure being developed by 50+ developers using Maven 2/3 with Nexus and Jenkins. We develop at a fast pace and more often that not we build and deploy releases with SNAPSHOT dependencies. Most of the time we cannot wait for one team or another to build a release version before we have to deploy. Due to this we have had a number of issues with bugs and works-in-process entering production and having no idea what changes are in those SNAPSHOTS.

What I am looking to do is to eliminate SNAPSHOTS and move to automatic releases with the versions-maven-plugin and to implement a release Staging policy.

And here in lays the problem. For developers and for the CI build it needs to be configured to resolve dependencies from "Staging" and "Release" and publish ONLY to "Staging". We then want to be able to "Promote" that build as a release, re-building it to resolve dependencies from "Release" and publish to "Release" (initially this will be a manual promotion but we may wish to automate this also). Does this all sound reasonable?

I do not want to use the maven-release-plugin because we have tried to use this before and we don't like the fact that it modifies the pom automatically and makes changes in our scm, triggering more builds.

Also, is there even a way to tell maven to use one repository for resolving and another to publish to? Could I do this with gradle?

Any comments/ideas on this would be greatly appreciated.

Upvotes: 2

Views: 589

Answers (3)

Cengiz
Cengiz

Reputation: 4867

I recommend not to use the maven release plugin and not to use snapshots. Look at here to release in a fast and clean way.

Upvotes: 1

Matt
Matt

Reputation: 8476

It sounds a lot like my build system.

I do this using gradle and 2 separate nexus repositories (both "release" repositories, 1 staging and 1 final). The build has the concept of a target maturity and it uses this to calculate the version number (using semantic versioning) so an RC build produces a version like 1.0.0-rc1 and a final build produces a version like 1.0.0. We then have tests tagged in different ways (testng groups, cucumber tags etc) so that different slices of the tests run at different times. The build decides which repositories to depend on or publish to based on the target maturity and hence can ensure that only sufficiently mature artefacts are consumed.

This is configured to run automatically via teamcity build chains so a commit in some core lib ripples through the downstream builds and on to integration testing/deployment (by rundeck via its java api by a set of gradle tasks) if necessary.

Gradle publication and resolution repositories are configured separately so can be different. The same can be done in maven too.

For example, given a dependency graph like

corelib -> mylib -> myapp

Each thing would have a set of tests associated with them that are tagged as either beta or rc. The intention being that a build that produces an RC is one that has passed the beta tests (i.e. if you pass beta then you're mature enough to be an RC) & that build is one that finishes quickly (e.g. does unit tests only) whereas the rc tests (that produces a final build) might do some integration tests or some more long running tests. This definition is our own and is completely arbitrary, you could make any distinction you like. Ours is just based on applying increasingly rigorous and/or long lasting tests only once you have a certain confidence level in the product.

We then setup a build chain so that the rc builds depend on upstream rc builds and the final builds depend on the upstream final build and your own rc build

i.e.

             --> mylib final --
           /                   \
mylib rc --                     --> myapp final
           \                   /
             --> myapp rc -----

and so on. In this example the flow is

  • commit a change to mylib
  • mylib rc build runs
    • runs beta tests
    • publish result to rc repository
  • mylib final and myapp rc can run in parallel
  • mylib final build runs
    • runs rc tests
    • publish result to final repository
  • myapp rc build runs
    • depends on rc repository so picks up result of previous mylib rc build
    • runs beta tests
  • myapp final runs
    • depends on final repository so picks up result of previous mylib final build
    • runs rc tests

Version numbers at each point are calculated by interrogating source control

Dependencies, on our own artefacts, are dynamic (1.0.+ in ivy terms for example), major.minor is set statically (in source control) and the build is left to produce patch and candidate numbers itself, i.e. myapp 1.0 will depend on mylib 1.0.+ . It is much simpler IMO to have 2 separate repositories as the filtering mechanism rather than have to dig into the resolution logic in gradle/ivy to filter out the ones we didn't want.

Upvotes: 3

ptyx
ptyx

Reputation: 4164

Look at the maven-version-plugin, specifically the lock/unlock snapshot goals. They can help to identify which dependencies an assembly is using, and might be sufficient for you.

If not, yes - maven can use staging repositories, but it might lead to confusion if anything you want to stage is also pulled on local developers repositories.

What we're using right now to solve a similar problem is a 'scripted' somewhat equivalent to maven-release-plugin (it actually calls it for some operation), and an awkward versioning scheme: our development branch stays 1.2-SNAPSHOT, but our released/published builds use a different versioning scheme (1.2.3, 1.2.4). Our script tags git as well.

Upvotes: 2

Related Questions