CODEWITHSUNDEEP
azureazure-devopsazure-pipelinesazure-pipelines-yaml
Chris
Chris

Reputation: 1009

How to loop through user-defined variables in a YAML pipeline?

I am trying to loop through user-defined variables in an Azure DevOps YAML pipeline.
The variables have been created through the UI:

enter image description here

enter image description here

Below the YAML pipeline code that I'm using:

trigger:
- dev
- main

pr:
- dev

pool:
  vmImage: ubuntu-latest

stages:

- stage:
  jobs:
  - job: TestVars
    steps:
    - ${{ each var in variables }}:
      - script: |
          echo ${{ var.key }}
          echo ${{ var.value }}
        displayName: ${{ var.key }}

When running the above pipeline only system and build variables are listed (e.g. system, system.hostType, build.queuedBy, etc.).

Any help to loop through user-defined variables would be much appreciated.

Upvotes: 9

Views: 17840

Answers (3)

milo.farrell
milo.farrell

Reputation: 672

To expand on the awnser below. It is a bit round about but you can use the azure devopps CLI. This may be a bit overkill but it does do the job.

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- bash: az --version
  displayName: 'Show Azure CLI version'

- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject) --use-git-aliases true
  displayName: 'Set default Azure DevOps organization and project'

- bash: |
    az pipelines variable list --pipeline-id $(System.DefinitionId)
  displayName: 'Show build list varibales'
  env:
    AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

Build vars in pipeline

Build vars listed in pipeline run

This approach was taken from a combination of: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables and https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables

If the agent is self hosted you may need to install the dev opps cli.

Upvotes: 1

nicksnb
nicksnb

Reputation: 258

From the Microsoft docs link you provided, it specifies that:

"Unlike a normal variable, they are not automatically decrypted into environment variables for scripts. You need to explicitly map secret variables."

However, one workaround could potentially be to run an azure cli task and get the pipeline variables using az pipelines variable list

Assuming your intention is to get the actual values, in which case maybe that won't suffice. Having said that, you should consider a variable group even if you're not using them in other pipelines since the group can be linked to an Azure KeyVault and map the secrets as variables. You can store your sensitive values in a KeyVault and link it to the variable group which can be used like regular variables in your pipeline.

Or you can access KeyVault secrets right from the AzureKeyVault pipeline task.

Upvotes: 4

YK1
YK1

Reputation: 7602

Unfortunately, no luck fetching the variables defined in UI. However, if your variables are non-secrets, you can bring them over into the YAML, and they will show up in the loop.

- stage:
  variables:
    myyamlvar: 1000   # this will show up in the loop
  jobs:
  - job: TestVars
    steps:
    - ${{ each var in variables }}:
      - script: |
          echo ${{ var.key }}
          echo ${{ var.value }}
        displayName: ${{ var.key }}

Alternatively, instead of using a compile time expression, you can list variables using a runtime construct, for example:

 - job: TestRuntimeVars
    steps:
      - script: |
          for var in $(compgen -e); do
            echo $var ${!var};
          done

This will list all variables including ones defined in the UI.

Upvotes: 8

Related Questions