use output variables from another stage in a different stage

Question

I have the next code snippets:

- task: AzureCLI@2
  name: A
  displayName: "Run tf ${{ parameters.action }}"
  inputs:
    azureSubscription: '***-${{ parameters.env }}' 
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      if [ ${{ parameters.action }} = "plan" ]; then
          echo "Running terraform plan..."
          timeout -s SIGINT 58m terraform plan out=terraform.plan
          changes=$(terraform show -no-color terraform.plan | grep -E "No changes.")
          echo "Checking for Changes variable [$changes]"
          if [ -n "$changes" ]; then
            echo "No changes detected. Exiting without further actions."
            echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'notdetected
          else
            echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'changesdetected
          fi
      else
          echo "Running terraform apply..."
          timeout -s SIGINT 58m terraform apply input=false auto-approve
      fi
    addSpnToEnvironment: true
    workingDirectory: '${{ parameters.workingDirectory }}'

These tasks are part of a template called script.yml and it is used in another template called stages.yml following the next structure in the next comment:

parameters:
- name: workingDirectory
  type: string
- name: environments
  type: object
  default:
  - dev
  - qa


stages:
- ${{ each env in parameters.environments }}:
    - stage: Plan_${{ env }}
      condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
      jobs:
        - job: Plan
          steps:
          - template: script.yml
            parameters:
              env: ${{ env }}
              action: plan
              workingDirectory: '${{ parameters.workingDirectory }}'

    - stage: Apply_${{ env }}
      dependsOn:
        - Plan_${{ env }}
      jobs: 
      - deployment: 
        displayName: Deploy
        environment: ${{ env }}
        strategy:
          runOnce:
            deploy:
              steps:            
              - template: script.yml
                parameters:
                  env: ${{ env }}
                  action: apply
                  workingDirectory: '${{ parameters.workingDirectory }}'

Now, I have isOutput=true option for (echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'notdetected), I need to consume this output variable in my Apply stage which is a different one. I need to set a condition based on this output for the Apply Stage to be skipped/to run only the variable output is "changesdetected"... but I tried first to consume that variable to see if works.

I have tried referencing this variable in the stages.yml: adding variables: varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ] or varPlan: $[ stageDependencies.Plan_${{ env }}.outputs['Plan.PlanOverview.PlanChanges'] ] when I run the script echo $(varPlan) not showing the output of variable.

According to: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops&branch=pr-en-us-1603#job-to-job-dependencies-across-stages:~:text=Dependency%20syntax%20overview

how to reference that variable output to my condition in the apply stage?

Tried like these: varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['AzureCLI@2.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['A.PlanChanges'] ]

varPlan: $[ dependencies.Plan_${{ env }}.outputs['Plan.AzureCLI@2.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.A.outputs['PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.PlanOverview.outputs['PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ]

Bright Ran-MSFT · Accepted Answer

On the stage-level of all the "Apply_xxx" stages, you can set the condition like as below.

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    . . .
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))

With this condition, the "Apply_xxx" stages will run when the value of output variable "PlanChanges" is "changesdetected".

In addition, another important thing you need to know is that based on the current definition in your main YAML (stages.yml), all the stages will run in a single line in sequence based on their order of defining in the YAML file. So, they will look like as below in a pipeline run.

In this situation, a stage will be skipped if any of the previous stages is not succeeded. For example, on above image, in the expected result, the "Apply_prod" stage should run but it was skipped due to the previous "Apply_qa" stage was skipped.

For your case, the ideal results should be:

"Plan_dev" and "Apply_dev" are in a separate line, and "Apply_dev" only depends on "Plan_dev".
"Plan_qa" and "Apply_qa" are in a separate line, and "Apply_qa" only depends on "Plan_qa".
"Plan_prod" and "Apply_prod" are in a separate line, and "Apply_prod" only depends on "Plan_prod".

To reach this, you can update the main YAML (stages.yml) like as below:

Add a "main" stage as the parent node of all the separate lines. In this "main" stage, you can let it do nothing. So, it will be always succeeded.
Set the "Plan_xxx" stage to depend on the "main" stage.
Set the "Apply_xxx" stage to depend on the "Plan_xxx" stage.

stages:
- stage: main
  jobs:
  - job: main
    steps:
    - checkout: none

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    dependsOn: main
    condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
    jobs:
    . . .
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))
    jobs:
    . . .

EDIT:

For the question in your first reply blow:

If I did not misnderstand your demands, when the value of output variable (PlanChanges) generated from 'Plan_${{ env }}' stage is 'changesdetected', then the corresponding 'Apply_${{ env }}' stage should run and not skip, and when the output value is 'notdetected', the 'Apply_${{ env }}' stage should skip.

So, if 'Plan_prod' stage outputs 'changesdetected', the 'Apply_prod' stage should run. On the first pipeline image I posted above, the 'Plan_prod' stage was outputting 'changesdetected', the 'Apply_prod' stage should not skip.
For the question in your second reply below:

You seem have wrong syntax when use the setvariable command to set the output variable using Bash.

Change the command line like as the following:
```
echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]notdetected"
echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]changesdetected"
```

EDIT_2:

Below I will share you with a sample that I attempted on my side. In this sample, I set the "Apply_qa" stage will be skipped based on the condition below.

condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))

You can reference this sample to check and update the code in your YAML files.

The main YAML file: azure-pipelines.yml

parameters:
- name: workingDirectory
  type: string
- name: environments
  type: object
  default:
  - dev
  - qa
  - prod

stages:
- stage: main
  jobs:
  - job: main
    steps:
    - checkout: none

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    dependsOn: main
    condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
    jobs:
    - job: Plan
      steps:
      - template: script.yml
        parameters:
          env: ${{ env }}
          action: plan
          workingDirectory: '${{ parameters.workingDirectory }}'
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))
    jobs:
    - deployment: Deploy
      environment: ${{ env }}
      variables:
      # The variable can be used by the steps (include script.yml) within this job. 
        varPlanChanges: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['A.PlanChanges'] ]
      strategy:
        runOnce:
          deploy:
            steps:
            - template: script.yml
              parameters:
                env: ${{ env }}
                action: apply
                workingDirectory: '${{ parameters.workingDirectory }}'

The template YAML file: script.yml

steps:
- task: Bash@3
  name: A
  displayName: 'Set Output'
  inputs:
    targetType: inline
    script: |
      if [ ${{ parameters.action }} = "plan" ]; then
        echo "The action is ${{ parameters.action }}."
        echo "The environment is ${{ parameters.env }}."
        if [ ${{ parameters.env }} = "qa" ]; then
          echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]notdetected"
        else
          echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]changesdetected"
        fi
      else
        echo "The action is not plan. It is ${{ parameters.action }}."
        echo "The environment is ${{ parameters.env }}."
      fi

- ${{ if eq(parameters.action, 'plan') }}:
  - task: Bash@3
    displayName: 'Variables for plan action'
    inputs:
      targetType: inline
      script: |
        echo "env = ${{ parameters.env }}"
        echo "action = ${{ parameters.action }}"
        echo "workingDirectory = ${{ parameters.workingDirectory }}"
        echo "PlanChanges = $(A.PlanChanges)"

- ${{ if eq(parameters.action, 'apply') }}:
  - task: Bash@3
    displayName: 'Variables for apply action'
    inputs:
      targetType: inline
      script: |
        echo "env = ${{ parameters.env }}"
        echo "action = ${{ parameters.action }}"
        echo "workingDirectory = ${{ parameters.workingDirectory }}"
        echo "varPlanChanges = $(varPlanChanges)"

The result of running the pipeline, see the second image I posted above.

use output variables from another stage in a different stage

Answers (2)

Related Questions