CODEWITHSUNDEEP
azure-devopsazure-devops-pipelines
BigPigVT
BigPigVT

Reputation: 1381

Azure DevOps yaml pipeline expression not evaluating

I'm trying to assign the PublishTestResults@2 task's failTaskOnFailedTests argument in an azure-pipelines.yml file to be the result of an expression. The first time the tests are run, if there are any failed tests I want to fail the job. On subsequent attempts to Retry failed jobs I don't want to fail the job.

I've set up the task like this (second to last line is where I'm setting the failTaskOnFailedTests argument):

    - task: PublishTestResults@2
      displayName: 'Publish Test Results'
      inputs:
        testResultsFormat: 'JUnit'
        testResultsFiles: '*.xml'
        searchFolder: '$(testsOutputPath)'
        mergeTestResults: true
        failTaskOnFailedTests: eq($(System.JobAttempt), 1)
        testRunTitle: 'Test Results $(System.JobAttempt)'

When I run the pipeline with system diagnostics enabled I can see this in the log:

##[debug]testRunTitle=Test Results 1
##[debug]publishRunAttachments=true
##[debug]failTaskOnFailedTests=eq(1, 1)
##[debug]searchFolder=/home/vsts/work/1/s/TestProject/cypress/reports/junit
##[debug]testRunner: JUnit
##[debug]testResultsFiles: *.xml
##[debug]mergeResults: true

The third line shows failTaskOnFailedTests being set to the expression statement, not the evaluated value of the expression. I'm at a loss for what I'm doing wrong. The expression seems in line with others in the Expressions documentation.

What am I missing?

Upvotes: 3

Views: 4012

Answers (1)

beatcracker
beatcracker

Reputation: 6920

In Azure DevOps you can use compile time (${{ <expressions> }}) and runtime ($[ <expression> ]) expressions.

Compile time expressions are processed once, when template is rendered by the server. Runtime expressions are evaluated when pipeline executes.

In your case you'd need to use runtime expression, but they can only be used in variable assignments or condition parameter.

Example:

condition: eq(variables['System.JobAttempt'], '1')

So using this approach you can't configure failTaskOnFailedTests input directly, but you can skip the task entirely.

I've not tested this, but to achieve the desired result you could try to use a script to set failTaskOnFailedTests depending on the System.JobAttempt value.

Example:

- bash: echo "##vso[task.setvariable variable=FailTask]$((( $(System.JobAttempt) > 1 )) && echo false || echo true)"

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '*.xml'
    searchFolder: '$(testsOutputPath)'
    mergeTestResults: true
    failTaskOnFailedTests: $(FailTask)
    testRunTitle: 'Test Results $(System.JobAttempt)'

There is also a chance that manipulating variable directly might work.

Example:

variables:
  FailTask: $[ le( variables['System.JobAttempt'], 1 ) ]

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '*.xml'
    searchFolder: '$(testsOutputPath)'
    mergeTestResults: true
    failTaskOnFailedTests: $(FailTask)
    testRunTitle: 'Test Results $(System.JobAttempt)'

Upvotes: 5

Related Questions