How to make gitlab run jobs in sequential order?

Question

I have a gitlab-ci.yml file like the following:

stages:
  - test
  - job1
  - job2


test:
  stage: test
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  script:
    ...

myjob1:
  stage: job1
  script:
    ...


myjob2:
  stage: job2
  script:
    ...

According to the documentation HERE (or at least how I understood it), the first stage/job is only run when I create a merge request.

This is true, but the next stage (job1) is run in parallel when the first job (test) has been started. As far as I understand the stages which are defined in the order test -> job1 -> job2) always run in sequence.

So what am I doing wrong? Why do job test and job1 run in parallel, and not in sequence as expected?

Answer 1

After a lot of trial-end-errors and reading and rereading parts of the really unclear and confusing documentation I might have found a solution.

First, the stage you only want to run on a merge request (or which you do not want to run if you schedule a trigger or start the pipeline manually), you need to change that stage as follows:

test:
  rules:
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - when: on_success
  stage: test
  script:
    - 'echo "Running Test"'
    - 'echo $CI_PIPELINE_SOURCE'

Here, you define a rule that checks if the variable CI_PIPELINE_SOURCE is either web or schedule. If the variable is web this means a manual pipeline trigger (i.e. you pressed manually on Run pipeline, which is not explained in the documentation), or if the pipeline is triggered by a schedule (not tested, but I assume that is what schedule means).

So if the pipeline is triggered by a scheduled event or manually, never tells gitlab to not execute that stage. The when: on_success is like an else statement, which tells gitlab to run that stage in any other case.

However, that is not the complete story. Because when you use git to make changes to the code and push it to gitlab via git push, you have two triggers in gitlab! A trigger merge_request_event and a trigger push. That means, the pipeline is started twice!

To avoid the pipeline started twice you need to use the workflow key, which helps to define if the pipeline (=workflow) is run or not. (The term workflow seems to mean pipeline). Here is the code to be put into the gitlab-ci.yml file:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: never
    - when: always

This construct suppresses the pipeline to be run when the trigger is merge_request_event. In that case, the additional pipeline is not run. In all other cases (when the trigger is e.g. push), the pipeline is run.

So here is the complete gitlab-ci.yaml code:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: never
    - when: always


stages:
  - test
  - stage1
  - stage2


test:
  rules:
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - when: on_success
  stage: test
  script:
    - 'echo "Running Test"'

my_stage1:
  stage: stage1
  script:
    - 'echo "Running stage 1"'


my_stage2:
  stage: stage2
  script:
    - 'echo "Running stage 2"'

If you make a git push then one pipeline is run with the stages test, my_stage1 and my_stage2, and when you start the pipeline manually or if it is triggered by a schedule, one pipeline is started with the stages my_stage1 and my_stage2.

As to why this is so complicated and confusing, I have not the slightest idea.