Choose approach for external Jenkins orchestration

Question

I wondering if there is a solution for external Jenkins orchestration or not?

Let me explain our setup first:

• We have several huge Java artifacts which needs to be tested through continuous integration pipeline
• A job is configured to grab a code after commit, run unit tests and build an artifact itself
• B jobs (there is 4 of them for now) run different set of smoke tests in parallel using empty databases. They triggered only after A job succeeded. Those jobs use external resources: cloud nodes (to which artifact will be deployed) and databases from DB pool. Jobs can return three statuses in Jenkins:
  • green: cloud node was obtained successfully, artifact was deployed, all tests passed
  • yellow: there is no infrastructure failures detected but some (or all) tests failed --> Blame developers/QA here
  • red: there was one one more infrastructure failure: cloud messed up, database unavailable, artifact deployment failed. --> Blame DevOps here
• C job: run only if all of B jobs were green. Otherwise end of flow. Uploading an artifact to Nexus
• D jobs: run functional tests: several jobs here will reuse cloud nodes from B jobs to save time for deployment. We don't care which status they will get: green, yellow or red. Just record results and that's it. However, there are two jobs here which will deploy original artifact from A job in clustered configuration and if deployment succeeded then E job start independently (i.e. if D1 job fail, than E1 shouldn't start but if D2 job succeeded in the same workflow, than E2 should be triggered).

When workflow is finished, you need to determine a status: if there was a fatal error (any red build) than workflow must have a failed status.

After workflow if finished, all resources must be freed up: cloud nodes and databases.

Which workflow/process flow tool or library can I utilize to implement such logic?

We have several other pipelines with different flows so it should be flexible.

Current implementation have very complicated logic and several technical jobs which will take care of shared resources. We want to implement that logic in single place: an orchestrator which will be triggering Jenkins jobs externally, feeding them with parameters to run.

Answer 1

Checkout the Rundeck plugin for jenkins.

Rundeck is an SSH based orchestration tool.