Azure pipeline with custom variables as condition

Question

We would like to optimize our pipelines to avoid running steps that are not needed for the particular case.
I have created the variable shouldTriggerAnyBuild but it always seems to be true (or ignored), as the specified steps always runs, even though the none of the steps later, where the conditions are combined from, are run.

What is the problem with the script, or how can I debug it?

trigger:
- master
- stage
- release/*

pool:
  vmImage: 'macOS-latest'

variables:
  shouldTriggerAnyBuild: $[ or(and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))), eq(variables['Build.SourceBranch'], 'refs/heads/stage'), startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')) ]
  
steps:
- task: UseRubyVersion@0
  inputs:
    versionSpec: '~> 2.6'

- script: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager --install 'system-images;android-29;google_apis;x86'
  displayName: 'install/setup android sdkmanager'
  condition: variables.shouldTriggerAnyBuild

- script: gem install bundler
  displayName: 'gem install bundler'
  condition: variables.shouldTriggerAnyBuild

- script: bundle install
  displayName: 'bundle install'
  condition: variables.shouldTriggerAnyBuild

- script: bundle exec fastlane ciBuildDev
  displayName: 'build dev'
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/master'),  not(startsWith(variables['Build.SourceVersionMessage'], 'release:')))

- script: bundle exec fastlane ciDeployToTest
  displayName: 'build stage'
  condition: eq(variables['Build.SourceBranch'], 'refs/heads/stage')

- script: bundle exec fastlane ciDeployToGooglePlay
  displayName: 'build release'
  condition: startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')
    
- task: PublishBuildArtifacts@1
  displayName: "Publish artifacts .apk"
  condition: startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')
  inputs:
    PathtoPublish: ./app/build/outputs/apk/prod/app-prod-unsigned.apk
    ArtifactName: Prod_app_apk
        

https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops

Answer 1

After a lot of trial and error we found that there was Build.SourceVersionMessage that meant we could not use it as a variable, so we ended up with a hack as shown below:

trigger:
  - master
  - stage
  - release/*

pool:
  vmImage: "macOS-latest"

variables:
  # we can't use Build.SourceVersionMessage up here because it's not defined when the variables is set here

  isMaster: ${{eq(variables['Build.SourceBranch'], 'refs/heads/master')}}
  isStage: ${{eq(variables['Build.SourceBranch'], 'refs/heads/stage')}}
  isRelease: ${{startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')}}
  releaseMessagePrefix: "release:"

jobs:
  - job:
    steps:
      - task: UseRubyVersion@0
        displayName: "set ruby version"
        inputs:
          versionSpec: "~> 2.6"

      - task: Bash@3
        displayName: "set commitMessage variable"
        inputs:
          targetType: inline
          script: echo '##vso[task.setvariable variable=commitMessage]$(Build.SourceVersionMessage)'

      - script: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager --install 'system-images;android-29;google_apis;x86'
        displayName: "install/setup android sdkmanager"
        condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})

      - script: gem install bundler
        displayName: "gem install bundler"
        condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})

      - script: bundle install
        displayName: "bundle install"
        condition: or(and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix))), ${{variables.isStage}}, ${{variables.isRelease}})

      - script: bundle exec fastlane ciBuildDev
        displayName: "Build: dev"
        condition: and(variables.isMaster, not(startsWith(variables.commitMessage, variables.releaseMessagePrefix)))

      - script: bundle exec fastlane ciDeployToTest
        displayName: "Build: stage"
        condition: ${{variables.isStage}}

      - script: bundle exec fastlane ciDeployToGooglePlay
        displayName: "Build: release"
        condition: ${{variables.isRelease}}

      - task: PublishBuildArtifacts@1
        displayName: "Publish: .apk"
        condition: ${{variables.isRelease}}
        inputs:
          PathtoPublish: ./app/build/outputs/apk/prod/app-prod-unsigned.apk
          ArtifactName: Prod_app_apk

Answer 2

What is the problem with the script?

Conditions are written as expressions. condition: variables.shouldTriggerAnyBuild won't take effect. Instead you can use condition: eq(variables['SHOULDTRIGGERANYBUILD'], 'True') for conditional step. I think it's the direct cause of your issue. Also, feel free to use condition: eq(variables['shouldTriggerAnyBuild'], 'True') if you want, it also works.

How to debug it?

Here's a quick way to debug the value when necessary:

1.Change the second value to a impossible variable, then you can check the real value for your custom variable when the step is skipped:

2.If you want to make all thing clear, then you can do something like:

- task: CmdLine@2
  inputs:
    script: |
      echo Hello world
  condition: eq(and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))),'True')

- task: CmdLine@2
  inputs:
    script: |
      echo Hello world
  condition: eq(eq(variables['Build.SourceBranch'], 'refs/heads/stage'),'True')

- task: CmdLine@2
  inputs:
    script: |
      echo Hello world
  condition: eq(startsWith(variables['Build.SourceBranch'], 'refs/heads/release/'),'True')

Since your variable is the combination of and(eq(variables['Build.SourceBranch'], 'refs/heads/master'), not(startsWith(variables['Build.SourceVersionMessage'], 'release:'))), eq(variables['Build.SourceBranch'], 'refs/heads/stage') and startsWith(variables['Build.SourceBranch'], 'refs/heads/release/') via Or function, you can divide them to debug how the shouldTriggerAnyBuild is expanded as always True.

In this way, you can debug them easily to find how the variable is expanded:

Note:

1.Or function: Evaluates True if any parameter is true.

2.Most of the time we choose to skip current task if one of previous tasks have failed, so you can consider combining succeeded() and your variable shouldTriggerAnyBuild. Examples here.