CODEWITHSUNDEEP
gitlabgitlab-ci
Chad Quinlan
Chad Quinlan

Reputation: 41

GitLab CI/CD - Using Both Includes: and Needs:

I'm working off the Auto-Devops template for my .gitlab-ci.yml. Trying to use both include: and needs: as part of a GitLab partner lab. The CI Lint tool says this is valid, but the pipeline fails, saying "dast: needs 'dast_environment_deploy'". After attempting the below code, I even tried copying the content of the entire dast_environment_deploy template and placing that in the file, still getting the same error.

How do I get my pipeline file to use needs: based on an include: template?

image: alpine:latest

stages:
  - build
  - test
  - scan
  - deploy  # dummy stage to follow the template guidelines
  - review
  - dast
  - staging
  - canary
  - production
  - incremental rollout 10%
  - incremental rollout 25%
  - incremental rollout 50%
  - incremental rollout 100%
  - performance
  - cleanup

scan:
    stage: scan
    trigger:
        include:
            - template: Security/License-Scanning.gitlab-ci.yml
            - template: Security/Container-Scanning.gitlab-ci.yml
            - template: Security/Dependency-Scanning.gitlab-ci.yml

review:
  needs: ["build"]

dast:
  needs: ["dast_environment_deploy"]

sast:
  needs: []

cache:
    paths:
        - node_modules

include:
  - template: Jobs/Build.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml
  - template: Jobs/Test.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Test.gitlab-ci.yml
  - template: Jobs/Code-Quality.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Code-Quality.gitlab-ci.yml
  - template: Jobs/Code-Intelligence.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Code-Intelligence.gitlab-ci.yml
  - template: Jobs/Deploy.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
  - template: Jobs/Deploy/ECS.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml
  - template: Jobs/Deploy/EC2.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/EC2.gitlab-ci.yml
  - template: Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml
  - template: Jobs/Browser-Performance-Testing.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml
  - template: Security/DAST.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml
  - template: Security/SAST.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml

Upvotes: 4

Views: 5096

Answers (1)

Adam Marshall
Adam Marshall

Reputation: 7649

I haven't reviewed each of your included templates, but based on the error and the few I did review, the error is most likely caused by the needs keyword needing a job that isn't added to the pipeline due to a when condition or rules:if condition.

If a job needs another job, and the other job isn't added to the pipeline (the actual running pipeline instance, not the pipeline definition in .gitlab-ci.yml), the yml is considered invalid at runtime. You can see all of the requirements and limitations with needs in the docs: https://docs.gitlab.com/ee/ci/yaml/#requirements-and-limitations

Looking at the first included template, - template: Jobs/Build.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml, both the build and build_artifact jobs have rules that could result in the job not being added to the pipeline. For example, if the variable $AUTO_DEVOPS_PLATFORM_TARGET is not "EC2" neither job will be added, so any job that needs these jobs will throw a YML error.

Upvotes: 1

Related Questions