CODEWITHSUNDEEP
azure-devopsyamlazure-pipelines
k_rollo
k_rollo

Reputation: 5472

Rerun failed tests in VSTest task using YAML

I want to rerun failed tests in Azure Pipelines via YAML.

Project structure:

RerunTestsDemo.sln
  +-- ClassLibrary1.csproj
  +-- UnitTestProject1.csproj
  +-- UnitTestProject2.csproj

azure-pipelines.yml:

trigger:
- automation/infra

pool: '$(AZURE_AGENT_POOL)'

steps:
- task: DotNetCoreCLI@2
  inputs:
    command: 'build'
    projects: '**/MyUnitTestProject.csproj'
- task: VisualStudioTestPlatformInstaller@1
  inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'latestStable'
- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\** 
    searchFolder: '$(System.DefaultWorkingDirectory)'
    publishRunAttachments: true
    rerunFailedTests: true
    rerunType: 'basedOnTestFailurePercentage'
    rerunFailedThreshold: '30'
    rerunMaxAttempts: '2'
  env:
    # All env vars correctly configured in pipeline

But it is not finding any test:

enter image description here

Upvotes: 0

Views: 4909

Answers (2)

k_rollo
k_rollo

Reputation: 5472

I eventually figured it out. The test assembly task was not matching the project pattern.

This is the barebones working YAML:

trigger:
- master

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*Tests*.sln'

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'

- task: VisualStudioTestPlatformInstaller@1
  inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'latestStable'

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\UnitTest*.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'
    testFiltercriteria: 'UnitTest1'
    publishRunAttachments: True
    rerunFailedTests: True
    rerunType: 'basedOnTestFailureCount'
    rerunFailedTestCasesMaxLimit: '5'
    rerunMaxAttempts: '2'

Output:

enter image description here

Upvotes: 0

Bright Ran-MSFT
Bright Ran-MSFT

Reputation: 13599

Please try the following to see if it can work.

steps:
- task: DotNetCoreCLI@2
  displayName: 'dotnet build $(buildConfiguration)'
  inputs:
    projects: |
      **/ClassLib1.csproj
      **/ClassLib2.csproj
    arguments: '--configuration $(buildConfiguration)'

- task: DotNetCoreCLI@2
  displayName: 'dotnet test $(buildConfiguration)'
  inputs:
    command: test
    projects: '**/MyUnitTestProject.csproj'
    arguments: '--configuration $(buildConfiguration)'

[UPDATE]

Here is a sample as reference:

steps:
. . .

- task: VSBuild@1
  displayName: 'Build solution **\*.sln'
  inputs:
    solution: '$(Parameters.solution)'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\** 
    searchFolder: '$(System.DefaultWorkingDirectory)'
    publishRunAttachments: true
    rerunFailedTests: true
    rerunType: 'basedOnTestFailurePercentage'
    rerunFailedThreshold: '30'
    rerunMaxAttempts: '2'

Upvotes: 2

Related Questions