CODEWITHSUNDEEP
jenkinsjenkins-pipelinejenkins-plugins
Vitaly Karasik DevOps
Vitaly Karasik DevOps

Reputation: 508

How to prevent concurrent execution of jobs on the same node, but allow a few parallel steps execution

I have multibranch job, whole pipeline runs on certain node. Currently I have a slaves with single executor, so multiple jobs cannot run on the same node. I'd like to use parallel steps inside my pipeline, so I need to increase number of executors. What is the best way to prevent concurrent job execution? Such configuration was trivial with Jenkins freestyle job and Throttling plugin - I simple used "Maximum Concurrent Builds Per Node". But in pipeline it seems that I cannot use "throttle" for whole pipeline, I need to use it for each step. Is there more elegant solution?

TIA, Vitaly

Upvotes: 1

Views: 3076

Answers (1)

biruk1230
biruk1230

Reputation: 3154

If I understand you correctly, you can just increase a number of executors for your node and use Do not allow concurrent builds option (in General section of the job) for all your jobs including pipelines. With this option enabled you still will be able to run parallel steps inside your pipeline (just cannot run concurrent builds of that pipeline).

If you still want to allow concurrent builds option, try to configure global throttle category (in Manage Jenkins -> Configure System page) and then use it in your pipeline, like in the example from the plugin documentation (you just need to wrap your parallel builds into throttle() section):

// The script below triggers 6 subtasks in parallel.
// Then tasks will be throttled according to the category settings.
def labels = ['1', '2', '3', '4', '5', '6'] 
def builders = [:]
for (x in labels) {
    def label = x // Need to bind the label variable before the closure 

    // Create a map to pass in to the 'parallel' step so we can fire all the builds at once
    builders[label] = {
      node('linux') {
        sh "sleep 5"
      }
    }
}

throttle(['myThrottleCategory1', 'myThrottleCategory2']) {
  parallel builders
}

Note: seems to be that such possibility is available only for scripted pipelines.

Upvotes: 1

Related Questions