CODEWITHSUNDEEP
gitlabgitlab-cigitlab-ci-runner
Yogesh
Yogesh

Reputation: 303

Gitlab - How to use same runner for entire pipeline execution

The pipeline is having below stage.

  1. download - It will download project zip file from artifactory depends on the release number
  2. build - It will compile and deploy the downloaded zip file on runner.
  3. test - It will execute the tests and upload the result.

To execute the above pipeline, I am having multiple Gitlab runners for each environment ( dev, test, prod) with same tag name e.g. I am having 2 runners in dev environment with tag linux git-runner-dev-1, git-runner-dev-2

Consider, on same time Multiple users trigger the pipeline in dev environment. Then the pipeline execution get started. But the entire pipeline will not get executed on same runner. For each stage it will select the runner which is available and thats why I am not getting the required output from the pipeline. I want to execute the entire pipeline on same runner which starts the execution of download stage.

Below sample yml file has been given for the reference.

stages:
  - download
  - build
  - test

download:
  stage: download
  script:
    - echo "===== Download starts ====="
    - echo "===== Download Ends ====="
  tags:
    - linux
build:
  stage: build
  script:
    - echo "===== Build starts ====="
    - echo "===== Build Ends ====="
  tags:
    - linux
test:
  stage: test
  script:
    - echo "===== Test starts ====="
    - echo "===== Test Ends ====="
  tags:
    - linux

Upvotes: 1

Views: 3129

Answers (1)

KMZ
KMZ

Reputation: 606

You can't do that (at least not with the current Gitlab versions).

This could be explained as "works as designed" — you probably want to parallelize your pipelines in general case, otherwise you face the risk of congestion and delays.

There is a couple of Gitlab CI features you can use to still make your pipeline work even when jobs are executed by different runners:

  • use caching
  • use job artifacts
  • use external storage shared between your runners to pass results from one runner to the next (NOTE: make sure your extrnal storage is not part of the $CI_PROJECT_DIR, otherwise Gitlab might decide to clean it before starting another job).

And make sure you understand the difference between caching and artifacts.

If you absolutely need all jobs in the pipeline to be executed by the same runner, you have the following options:

  • tag your runners with exclusive tags or combination of tags, and use those tags in job definitions to enforce a particular runner:
stages:
  - download
  - build
  - test

download:
  stage: download
  script:
    - echo "===== Download starts ====="
    - echo "===== Download Ends ====="
  tags:
    - linux_runner1
build:
  stage: build
  script:
    - echo "===== Build starts ====="
    - echo "===== Build Ends ====="
  tags:
    - linux_runner1
test:
  stage: test
  script:
    - echo "===== Test starts ====="
    - echo "===== Test Ends ====="
  tags:
    - linux_runner1
  • rewrite your pipeline to run all script commands in the same job instead of having them split into multiple jobs:
build:
  stage: build
  script:
    - echo "===== Download starts ====="
    - echo "===== Download Ends ====="
    - echo "===== Build starts ====="
    - echo "===== Build Ends ====="
    - echo "===== Test starts ====="
    - echo "===== Test Ends ====="
  tags:
    - linux
  • (variant of the above) put all script commands into a real script file (either as part of the image the runner uses, or part of the repo you're building) and execute that script instead.

Upvotes: 3

Related Questions