CODEWITHSUNDEEP
continuous-integrationgitlabsharedrunner
maxgemilian
maxgemilian

Reputation: 143

How to stick with a Gitlab CI Runner within a pipeline?

In our Gitlab project group, we are using multiple shared runners for CI. However, some of the jobs have dependencies, such that the previous job must have been executed on the same runner.

Here is an Example:

Now, with multiple shared runners it may happen, that job1 is executed on runner 1 and job 2 and 3 on a different runner than runner 1. This throws an error in job 2 and 3 as the docker image is locally not available on that runner.

On the other hand, we need multiple runners due to the amount of computation in our project. So it would be great if once a runner is picked in a specific job, it keeps the same runner for the ongoing jobs.

Any ideas to solve this?

Upvotes: 5

Views: 2899

Answers (1)

Thomas B
Thomas B

Reputation: 63

Gitlab scheduling doesn't permit it easily. The balancing between runners work as the following:

  • When a job is created on Gitlab instance, it registers a job in pending state
  • Runners check every 3 seconds (3s by default, configured with check_interval) if there are jobs in the pending queue. If yes, and if the runner is the right runner for the job (for example if job tags are compliant with runner), then the runner launch N jobs from the queues, N limited by the maximum number of concurrent job per runner (concurrent option)

So this isn't Gitlab itself that schedule which runner runs which job. Gitlab just put jobs to run in a queue, runners frequently check the queue and select jobs. It's great for scalability but not great for your use case.

In my point of view, you have 2 options:

Upvotes: 2

Related Questions