Using pipeline variables across stages with template jobs

Question

Problem Description

I was having some problems trying to use variables created in one stage in another stage and managed to find various articles old and new describing how this can be done. The more recent articles/posts identifying the new syntax

$[stageDependencies.{stageName}.{jobName}.outputs['{stepName}.{variableName}']

Used like this:

variables:
  myVariable: $[stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage']]

This works great until you needed to use job templates.

None of the samples I found online covered the situation of templates. They just demonstrated how multiple stages in the same yaml file could obtain the value.

The syntax depends on being able to put the expression into a variable. Unfortunately, when you use a template for a job, it's not possible to declare variables and passing it as a parameter results in it being unevaluated.

- stage: UseJobTemplateStage
  displayName: 'Use Job Template Stage'
  dependsOn: CreateStageVarStage
  jobs:
  - template: templates/job-showstagevars.yml
    parameters:
      ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]

In this snippet, it comes through as-is. The value does not get substituted in.

Theoretically, you could set your job to have the expression present in the variables block but that sort of hard-coding undermines one of the main benefits of templates.

Related Articles

Share variables across stages in Azure DevOps Pipelines
Azure DevOps Release Notes - sprint 168

Answer 1

Solution

The answer isn't actually far away. The original expression just need to be passed through a variable in the template job. Basically, set a variable to be the value of the parameter and use the macro syntax to evaluate the variable.

parameters:
- name: ValueToOutput
  type: string

...

  variables:
  - name: LocalVarOfValueToOutputParam
    value: ${{ parameters.ValueToOutput }}

Using the macro syntax of $(LocalVarOfValueToOutputParam) will result in the value making its way into the template job correctly.

Few other pointers:

• if you have a matrix-strategy defined you will need to add the Strategy in ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['Strategy.SetValueStep.VariableFromFirstStage'] ]
• if you are using script instead of powershell, you have to use true without $ in echo "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=true]$message"
• if you're not doing this in between stages, but in between jobs, use $[ dependencies.CreateStageVarStage.CreateStageVarJob.outputs['Strategy.SetValueStep.VariableFromFirstJob'] ]

Example

If we have a yaml file for the build definition:

stages:
- stage: CreateStageVarStage
  displayName: 'Create StageVar Stage'
  jobs:
  - job: CreateStageVarJob
    displayName: 'Create StageVar Job'
    timeoutInMinutes: 5
    pool:
      name:    'Azure Pipelines'
      vmImage: 'windows-2019'
    steps:
      - checkout: none 
      - pwsh: |
          [string]$message = 'This is the value from the first stage'
          Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
          Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
        name: SetValueStep

- stage: UseJobTemplateStage
  displayName: 'Use Job Template Stage'
  dependsOn: CreateStageVarStage
  jobs:
  - template: templates/job-showstagevars.yml
    parameters:
      ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]

That uses the job template templates/job-showstagevars.yml

parameters:
- name: ValueToOutput
  type: string

jobs:
- job: ShowStageVarJob
  displayName: 'Show stage var'
  timeoutInMinutes: 5
  pool:
    name:    'Azure Pipelines'
    vmImage: 'windows-2019'

  variables:
  - name: LocalVarOfValueToOutputParam
    value: ${{ parameters.ValueToOutput }}

  steps:
  - checkout: none
  - pwsh: |
      Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
      Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
      Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
    displayName: 'Show StageVariable'

What we get in our output of the second stage is this. Note how only the last expression evaluates to the correct value!

ValueToOutput parameter=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (pre-processor syntax)=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (macro syntax)=This is the value from the first stage

Answer 2

I know the question asks about template jobs, but for future reference I want to describe how it can be achieved with template stages as well.

It is done with a variable workaround as in the accepted answer, and with a reference to stagedependencies when no dependsOn exists. (Templates don't allow dependsOn). Somehow, this still works.

Example YAML using stage template (I have modified the code from the accepted answer):

stages:
- stage: CreateStageVarStage
  displayName: 'Create StageVar Stage'
  jobs:
  - job: CreateStageVarJob
    displayName: 'Create StageVar Job'
    timeoutInMinutes: 5
    pool:
      name:    'Azure Pipelines'
      vmImage: 'windows-2019'
    steps:
      - checkout: none 
      - pwsh: |
          [string]$message = 'This is the value from the first stage'
          Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
          Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
        name: SetValueStep

# stage template cannot use dependsOn, but is still allowed to refer to stagedependencies...
- template: templates/stage-showstagevars.yml
  parameters:
      ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]

Stage template:

parameters:
- name: ValueToOutput
  type: string

stages:
- stage: ShowStageVarStage

variables:
- name: LocalVarOfValueToOutputParam
  value: ${{ parameters.ValueToOutput }}

jobs:
- job: ShowStageVarJob
  displayName: 'Show stage var'
  pool:
    name:    'Azure Pipelines'
    vmImage: 'windows-2019'

  steps:
  - checkout: none
  - pwsh: |
      Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
      Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
      Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
    displayName: 'Show StageVariable'