CODEWITHSUNDEEP
global-variablesgitlab-ci
Joao Vitorino
Joao Vitorino

Reputation: 3256

set up global variables dynamically in gitlab-ci

I want to set some variables by getting the values from a pom.xml file. These variables need to be global because they will be used in multiple stages and jobs.

According to the gitlab-ci documentation, I can set global variables in two differents ways:

  1. using a variable statement:

    variable:  
 pom_artifactID: $(grep -m1 '<artifactId>' pom.xml | cut -d '<' -f2  |cut -d '>' -f2)

  2. Using a "before" script:

     before_script:
   - pom_artifactID=$(grep -m1 '<artifactId>' pom.xml | cut -d '<' -f2  |cut -d '>' -f2)
   - pom_artifactVersion=$(grep -m1 '<version>' pom.xml | cut -d '<' -f2  |cut -d '>' -f2)
   - pom_packaging=$(grep -m1 '<packaging>' pom.xml | cut -d '<' -f2  |cut -d '>' -f2)
   - pom_finalName=$({ grep -m1 '<finalName>' pom.xml |  cut -d '<' -f2 | cut -d '>' -f2; [ ${PIPESTATUS[0]} -eq 0 ] && true || echo ${pom_artifactID}-${pom_artifactVersion}.$pom_packaging}; })

The first doesn't work because gitlab-ci doesn't evaluate $(command), so pom_artifactID becomes a literal "$(grep -m1 '' pom.xml | cut -d '<' -f2 |cut -d '>' -f2)"

The second doesn't work either because "before_script" relies on the "grep" command and some docker images used in my pipeline have an old version of grep.

There is another way to set global variables o pass variables between stages and jobs?

Upvotes: 22

Views: 55181

Answers (3)

Joao Vitorino
Joao Vitorino

Reputation: 3256

Since gitlab 13.0, is possible to save the variables in a dotenv file format that is loaded in other stages.

build-job:
  stage: build
  script:
    - echo "BUILD_VARIABLE=value_from_build_job" >> build.env
  artifacts:
    reports:
      dotenv: build.env

test-job:
  stage: test
  script:
    - echo "$BUILD_VARIABLE"  # Output is: 'value_from_build_job'

Still very similar to the accepted answer but now is not necessary explicitly load the file with the variables before a new stage/job starts but if you want you can use needs or dependencies keywords to do so.

Upvotes: 6

Ben A
Ben A

Reputation: 522

There is another option that I have used as well. You can setup variables in the CI/CD settings, then from within a job issue a PUT back to the API to set them to a value you want to share across jobs.

script:
- >- 
  curl --request PUT --header "PRIVATE-TOKEN: $GITLAB_PROJ_TOKEN" 
  "https://someinstance.gitlab.com/api/v4/projects/$CI_PROJECT_ID/variables/MY_VARIABLE"
  --form "value=0"

In the above example it is calling a custom instance of gitlab, I also have a project access token defined in the project as GITLAB_PROJ_TOKEN. My variable name is MY_VARIABLE.

Upvotes: 6

Ivan
Ivan

Reputation: 9675

Passing values between jobs and stages

There is currently no way in GitLab to pass environment variable between stages or jobs.
But there is a request for that: https://gitlab.com/gitlab-org/gitlab/-/issues/22638

Current workaround is to use artifacts - basically pass files.
We had a similar use case - get Java app version from pom.xml and pass it to various jobs later in the pipeline.

How we did it in .gitlab-ci.yml:

stages:
  - build
  - package

variables:
  VARIABLES_FILE: ./variables.txt  # "." is required for image that have sh not bash

get-version:
  stage: build
  script:
    - APP_VERSION=... # $CI_COMMIT_TAG
    - echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
  artifacts:
    paths:
      - $VARIABLES_FILE
package:
  stage: package
  script:
    - source $VARIABLES_FILE
    - echo "Use env var APP_VERSION here as you like ..."

 
 

Extracting values from pom.xml

By the way it's better to treat xml.pom as XML to extract values from pom.xml rather than plain grep, because XML elements can potentially span multiple lines.

There are at least a couple of options, examples:

  1. Use XPath in xmllint tool from libxml2-utils
get-version:
  image: ubuntu
  script:
    - apt-get update
    - apt-get install -y libxml2-utils
    - APP_VERSION=`xmllint --xpath '/*[local-name()="project"]/*[local-name()="version"]/text()' $POM_FILE`
  1. Use python xml processing
get-version:
  image: python3
  script:
    - APP_VERSION=$(python3 -c "import xml.etree.ElementTree as ET; print(ET.parse(open('pom.xml')).getroot().find('{http://maven.apache.org/POM/4.0.0}version').text)")

Upvotes: 34

Related Questions