CODEWITHSUNDEEP
gitlab
Ashish Pratap
Ashish Pratap

Reputation: 479

Reducing build pipeline time using parallel keyword in Gitlab?

I am trying to reduce the the Gitlab pipeline execution time, by running multiple thread for a job using “parallel: 4” keyword in one of my bottleneck build job. With this I can see that my job as 4 threads with name like 1/4 to 4/4, but the execution is not parallel, they execute sequentially, can someone help me understand why this is happening and any other way to reduce the build time by running things in parallel?

Upvotes: 0

Views: 843

Answers (1)

Simon Schrottner
Simon Schrottner

Reputation: 4764

This is a tricky question to answer - because the best answer is it depends.

Also the documentation about the Parallel Directive states

Jobs can only run in parallel if there are multiple runners, or a single runner is configured to run multiple jobs concurrently.

Additionally it is also mentioned here again https://docs.gitlab.com/ee/ci/yaml/README.html#use-your-own-runners

  1. Do you run your own runners or do you use shared runners?

    The runners must be configured to allow concurrent execution, if they are not configured like that, you will not be able to run jobs in parallel. With own runners you might be able to change this.

    concurrent: limits how many jobs globally can be run concurrently. The most upper limit of jobs using all defined runners. 0 does not mean unlimited

  2. What is the limit of parallel execution on the runner?

    Besides the concurrent setting there is also the limit settings on the runner.

    limit: Limit how many jobs can be handled concurrently by this token. 0 (default) simply means don’t limit.

All this information about runners can be checked at the GitLab Documentation

Upvotes: 1

Related Questions