CODEWITHSUNDEEP
jenkinsbuildjenkins-pipeline
Ad N
Ad N

Reputation: 8376

How to nest dependent Jenkins Pipelines to execute on a single machine?

We are implementing Continuous Integration of several repositories using Jenkins.

For this example, let's assume libA is a dependency of libB which is a dependency of clientC.

libA pipeline

libA has external dependencies, so we can write the pipeline build-A-pipe to build it : one of the stages being responsible for gathering such dependencies, and a subsequent stage actually invoking the build command.

libB pipeline

libB would ideally be built within a separate pipeline, called build-B-pipe. In the stage to gather libB dependencies, we have to build libA. It seems to us that the best way to achieve such thing is to call build job: 'build-A-pipe' within the pipeline that builds libB (this way it allows to reuse the build-A-pipe, which already describes all steps required to successfully build libA).

clientC pipeline

Now, if we wanted to build clientC, we would follow a similar procedure. Thus, there would be a call like build job: 'build-B-pipe' in the dependencies gathering stage of the pipeline building clientC. The issue is that it results in nested calls to the build command, which deadlocks the single machine :

How should we approach this problem to make this inherently sequential build work within Jenkins ?

Upvotes: 2

Views: 1551

Answers (1)

Elliot Pryde
Elliot Pryde

Reputation: 90

The issue is that it results in nested calls to the build command, which deadlocks the single machine

By deadlock, do you mean that the slave agent which is responsible for executing the nested pipeline is running out of resources? Or is the node which is responsible for running these nested pipelines running out of executors?

If the machine responsible for running pipelines is exhausting all resources (assuming that this is the only responsibility of this machine), then your pipeline is too complex and should delegate more to other nodes/agents.

If the node is running out of executors, you can increase those in the node config.

Upvotes: 1

Related Questions