CODEWITHSUNDEEP
yamlazure-pipelines
PaulVrugt
PaulVrugt

Reputation: 1882

Pass variables to yaml template (like parameters) in azure pipelines

To minimize duplicate build scripts, we make use of templates in our pipelines. These templates have parameters. However, I now run into an issue, because I need to pass data to a template that is not yet available when the pipeline starts. (the data is generated in some steps during the pipeline). Since parameters are expanded when the pipeline starts, I cannot pass the data to the template via parameters.

I know I can reference output variables of different tasks, jobs, and stages within my pipeline, but the template I am using has no knowledge of the surrounding pipeline it is executed in. So the template doesn't know how to reference the output variables from other jobs outside of the template, simply because the template doesn't know what jobs have been executed before.

Is there some way I can map variables in my template? Ideally I would like to do something like this:

stages:
- stage: Stage1
  jobs:
  - some job that creates output variables
- stage: Stage2
  jobs:
  - template: 'myTemplate.yaml'
    variables:
      data1: $[ stageDependencies.Stage1.some_job.outputs['taskname.data1']]

and have the data1 variable available within the template.

So I am trying to avoid having to use: $[ stageDependencies.Stage1.some_job.outputs['taskname.data1']] in the template, because Stage1 might not even exist in all pipelines that uses the template.

I can actually do this, if the template only contains steps, instead of multiple jobs:

- stage: Stage2
  jobs:
  - job: Job1
    variables:
      data1: $[ stageDependencies.Stage1.some_job.outputs['taskName.data1']]

    steps:
    - template: templates/Azure/CreateTenant.yaml

Unfortunately, my templates contain multiple jobs

Update: I've entered a feature request to better support this situation: https://developercommunity.visualstudio.com/idea/1207453/yaml-template-variablesparameters-that-are-expande.html

Upvotes: 5

Views: 16060

Answers (2)

Ivan Samygin
Ivan Samygin

Reputation: 4571

In addition to the answer of Leo Liu-MSFT I'd like share two techniques I'm using to pass the YAML variables to templates:

  - stage: Stage2
    variables:
      entireJobOutputInJson: $[convertToJson(stageDependencies.Stage1.some_job.outputs)]

    jobs:
      - template: templates/Azure/CreateTenant.yaml
        parameters:
          variableNameWithJobOutputInJson: 'entireJobOutputInJson'

Key points:

  1. We can use convertToJson expression function to pass all output variables when needed.
  2. We have a parameter with a variable name, not the value. The variable name is just a static string value known at the template expansion moment, so we can pass it to the template. This technique allows to make the fact of using the external variable in the template visible. It allows to avoid issues during refactoring when variables are renamed, etc.

And in the template the variable can be expanded simply by the name with $(variableName) syntax:

parameters:
  variableNameWithJobOutputInJson: ''

jobs:
- job: Job1
  steps:
    - pwsh: |
        $obj = ConvertFrom-Json '$(${{ parameters.variableNameWithJobOutputInJson }})'
        $props = $obj.psobject.properties.name
        foreach ($property in $props ) {
          $varName = ($property -split "\.")[-1] # taking only the last part of multipart identifier
          echo "##vso[task.setvariable variable=$varName]$($obj.$property)"
        }
      displayName: 'Import Job output'

I used pwsh here because it works in both windows and linux based agents.

Upvotes: 2

Leo Liu
Leo Liu

Reputation: 76670

I can actually do this, if the template only contains steps, instead of multiple jobs. Unfortunately, my templates contain multiple jobs

To resolve this issue, you could define the variables at the stage level instead of the job level:

  - stage: Stage2
    variables:
      data1: $[ stageDependencies.Stage1.some_job.outputs['taskName.data1']]

    jobs:
      - template: templates/Azure/CreateTenant.yaml

Then we could avoid having to use: $[ stageDependencies.Stage1.some_job.outputs['taskname.data1']] in the template.

You could check the document Variable scopes for some more details.

Upvotes: 2

Related Questions