Conditional Stage Execution in Azure DevOps Pipelines

Question

I want a stage in an Azure DevOps pipeline to be executed depending on the content of a variable set in a previous stage.

Here is my pipeline:

stages:
  - stage: plan_dev
    jobs:
    - job: terraform_plan_dev
      steps:
      - bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
        name: terraform_plan

  - stage: apply_dev
    dependsOn: plan_dev
    condition: eq(stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'], '2')
    jobs:
    - deployment: "apply_dev"
      ...

The idea is to skip the apply_dev stage, if the plan_dev stage shows no changes. Background is that we have manual approval for the deployment in the plan_dev stage that we want to skip if there are no changes to be approved.

Unfortunately this doesn't seem to work. No matter whether the variable terraform_plan_exitcode is set with the expected value (2) or not, the apply_dev stage is skipped.

For the syntax, I followed the documentation here that says:

stageDependencies.StageName.JobName.outputs['StepName.VariableName']

Answer 1

TerraformTaskV2 has changesPresent output variable now, which can be used to skip apply stage.

• add name: to the plan task

stages:
  - stage: terraform_plan_STAGE
    jobs:
      - job: plan_JOB
        ...
        steps:
        ...
        - task: TerraformTaskV2@2
          name: 'plan_TASK'            # <===========
          displayName: 'plan'
          inputs:
            provider: 'azurerm'
            command: 'plan'
        ...

• add condition: to apply stage and check if changesPresent is true

    - stage: terraform_apply
      dependsOn: [terraform_plan]
      condition: eq(dependencies.terraform_plan_STAGE.outputs['plan_JOB.plan_TASK.changesPresent'], 'true')

reference:

1. https://github.com/microsoft/azure-pipelines-terraform/tree/main/Tasks/TerraformTask/TerraformTaskV2#output-variables
2. https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-outputs-in-a-different-stage

Answer 2

@Marius is correct. So this works

stages:
  - stage: plan_dev
    jobs:
    - job: terraform_plan_dev
      steps:
      - bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
        name: terraform_plan

  - stage: apply_dev
    dependsOn: plan_dev
    variables:
      varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]
    condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], 2)
    jobs:
    - job: apply_dev
      steps:
      - bash: echo 'apply $(varFromA)'
        name: terraform_apply

When you refer stage to stage dependencies you have different syntax

"dependencies": {
  "<STAGE_NAME>" : {
    "result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
    "outputs": {
        "jobName.stepName.variableName": "value"
    }
  },
  "...": {
    // another stage
  }
}

And when you refer to job to job across stage you have different syntax

"stageDependencies": {
  "<STAGE_NAME>" : {
    "<JOB_NAME>": {
      "result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
      "outputs": {
          "stepName.variableName": "value"
      }
    },
    "...": {
      // another job
    }
  },
  "...": {
    // another stage
  }
}

What is funny when you have job to job in one stage we use dependecies syntax again

"dependencies": {
  "<JOB_NAME>": {
    "result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
    "outputs": {
      "stepName.variableName": "value1"
    }
  },
  "...": {
    // another job
  }
}

This is a bit confusing and consider this in this as

• when you are on some level stage, job and refer to the same level from job to job or from stage to stage you have dependencies syntax
• when you want to refer from deeper level like from job to stage you should use stageDependencies

What is funny, in above example I used this on stage level:

variables:
      varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]

but this is evaluated at runtime and is evaluated from the job, so it is correct and is evaluated correctly.

I hope it added a value to previous answer.

Answer 3

I have seen this same issue. You need to use the dependencies variable instead of the stageDependencies:

stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
  steps:
  - bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
    name: terraform_plan

- stage: apply_dev
dependsOn: plan_dev
condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], '2')
jobs:
- deployment: "apply_dev"

The following is a more complete example of something I have working with Terraform Plan + conditional Apply:

stages: 
  - stage: Build_zip_plan
    displayName: Build portal, zip files and terraform plan
    jobs:
    - job: Build_portal_zip_files_terraform_plan
      pool:
        vmImage: 'ubuntu-latest'
      steps:
        - task: Cache@2
          displayName: 'Register TF cache'
          inputs:
            key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
            path: ${{ parameters.tfExecutionDir }}

        - task: TerraformInstaller@0
          displayName: 'Install Terraform'
          inputs:
            terraformVersion: ${{ parameters.tfVersion }}

        - task: TerraformTaskV1@0
          displayName: 'Terraform Init'
          inputs:
            provider: 'azurerm'
            command: 'init'
            workingDirectory: ${{ parameters.tfExecutionDir }}
            backendServiceArm: ${{ parameters.tfStateServiceConnection }}
            backendAzureRmResourceGroupName: ${{ parameters.tfStateResourceGroup }}
            backendAzureRmStorageAccountName: ${{ parameters.tfStateStorageAccount }}
            backendAzureRmContainerName: ${{ parameters.tfStateStorageContainer }}
            backendAzureRmKey: '$(prefix)-$(environment).tfstate'

        - task: TerraformTaskV1@0
          displayName: 'Terraform Plan'
          inputs:
            provider: 'azurerm'
            command: 'plan'
            commandOptions: '-input=false -out=deployment.tfplan -var="environment=$(environment)" -var="prefix=$(prefix)" -var="tenant=$(tenant)" -var="servicenow={username=\"$(servicenowusername)\",instance=\"$(servicenowinstance)\",password=\"$(servicenowpassword)\",assignmentgroup=\"$(servicenowassignmentgroup)\",company=\"$(servicenowcompany)\"}" -var="clientid=$(clientid)" -var="username=$(username)" -var="password=$(password)" -var="clientsecret=$(clientsecret)" -var="mcasapitoken=$(mcasapitoken)" -var="portaltenantid=$(portaltenantid)" -var="portalclientid=$(portalclientid)" -var="customerdisplayname=$(customerdisplayname)" -var="reportonlymode=$(reportonlymode)"'
            workingDirectory: ${{ parameters.tfExecutionDir }}
            environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}

        - task: PowerShell@2
          displayName: 'Check Terraform plan'
          name: "Check_Terraform_Plan"
          inputs:
            filePath: '$(Build.SourcesDirectory)/Pipelines/Invoke-CheckTerraformPlan.ps1'
            arguments: '-TfPlan ''${{ parameters.tfExecutionDir }}/deployment.tfplan'''
            pwsh: true
  

  - stage:
    dependsOn: Build_zip_plan
    displayName: Terraform apply
    condition: eq(dependencies.Build_zip_plan.outputs['Build_portal_zip_files_terraform_plan.Check_Terraform_Plan.TFChangesPending'], 'yes')
    jobs:
    - deployment: DeployHub
      displayName: Apply
      pool:
        vmImage: 'ubuntu-latest'
      environment: '$(prefix)'
      strategy:
        runOnce:
          deploy:
            steps:
            - checkout: self

            - task: Cache@2
              displayName: 'Get Cache for TF Artifact'
              inputs:
                key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
                path: ${{ parameters.tfExecutionDir }}
                
            - task: TerraformInstaller@0
              displayName: 'Install Terraform'
              inputs:
                terraformVersion: ${{ parameters.tfVersion }}

            - task: TerraformTaskV1@0
              displayName: 'Terraform Apply'
              inputs:
                provider: 'azurerm'
                command: 'apply'
                commandOptions: 'deployment.tfplan'
                workingDirectory: ${{ parameters.tfExecutionDir }}
                environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}