CODEWITHSUNDEEP
amazon-web-servicesgitlabgitlab-cibuild-automationgitlab-ci-runner
VIPIN PAL
VIPIN PAL

Reputation: 170

Gitlab CI pipeline to run jobs parallel in same stage and invoke/trigger other jobs of same stage

I am trying to create a automation pipeline for data load. I have a scenario as explained below:

stages
 - stage1
 - stage2

job1:
  stage: stage1
  script:
  - echo "stage 1 job 1"


job2:
  stage: stage1
  script:
   - echo "stage 1 job 2"

job3:
  stage: stage1
  script:
   - echo "stage 1 job 3"

job4:
  stage: stage1
  script:
   - echo "stage 1 job 4"

I want to run the job1 and job2 parallel in the same stage. So, after Job1 and job2 success

I am writing the pipeline in the .gitlab-ci.yml.

Can anyone help me to implement this?

Upvotes: 16

Views: 29439

Answers (1)

Mafor
Mafor

Reputation: 10681

Strict implementation of your requirements is not possible (according to my knowledge), the jobs 3 and 4 would need to be in a separate stage (although support for putting them in the same stage is planned). To be clear: the other functional requirements can be fulfilled, i.e:

  • job1 and job2 start in parallel
  • job1 will trigger the job3 (immediately, without waiting for the job2 to finish)
  • job2 will trigger the job4 (immediately, without waiting for the job1 to finish)

The key is using needs keyword to convert the pipieline to a directed acyclic graph:

stages:
    - stage-1
    - stage-2

job-1:
    stage: stage-1
    needs: []
    script: 
      - echo "job-1 started"
      - sleep 5
      - echo "job-1 done"

job-2:
    stage: stage-1
    needs: []
    script: 
      - echo "job-2 started"
      - sleep 60
      - echo "job-2 done"

job-3:
    stage: stage-2
    needs: [job-1]
    script: 
      - echo "job-3 started"
      - sleep 5
      - echo "job-3 done"

job-4:
    stage: stage-2
    needs: [job-2]
    script: 
      - echo "job-4 started"
      - sleep 5
      - echo "job-4 done"

pipeline

As you can see on the screenshot, the job 3 is started, even though the job 2 is still running.

Upvotes: 23

Related Questions