CODEWITHSUNDEEP
jenkinsjenkins-plugins
Carrie Garcia
Carrie Garcia

Reputation: 13

Jenkins - only run a job on a node with 2+ executors free

We have a number of multi-jobs that run a parent job and multiple sub-jobs. The parent does some preprocessing, then kicks off the first sub-job. Example:

Since the parent is running the entire time the sub-jobs are running, the process starts out with one executor, then picks up a second whenever a sub-job starts. drops it and then picks it back up when the next starts.

We have 4 nodes with 3 - 4 executors on each of them. We also don't have a networked drive so the sub job has to stay on the same executor as the parent to avoid having to pass the entire workspace between jobs.

The problem is that if one job is running and has two executors, then another gets kicked off and then another right after that, there's a chance they'll all end up on the same node and something like below happens:

Node 1

Now Parent2 and Parent3 just sit around waiting for a free executor. Eventually the Child job on Parent1 ends, then 2 r 3 grabs the executor and all of them are fighting for it.

Is there a way to tell Jenkins only kick off that parent on a node with at least 2 executors free? I know if someone started enough jobs quickly enough we could still end up with man issue, but it would significantly reduce the chance of it.

Upvotes: 0

Views: 1639

Answers (1)

Mor Lajb
Mor Lajb

Reputation: 2636

I think you can use - https://wiki.jenkins.io/display/JENKINS/Heavy+Job+Plugin , and define for each step the amount of free executers you need.

Upvotes: 1

Related Questions