Fail Azure DevOps pipeline .NET Core CLI test task when no tests are found

Question

Given a .NET Core CLI test task, specifically DotNetCoreCLI@2, that is expecting to discover tests, how can the task be configured to fail when no tests are discovered?

With the older Visual Studio Test task there were options for specifying this, which is what I'm basing this expectation on.

    failOnMinTestsNotRun: true
    minimumExpectedTests: '1'

Answer 1

As far as I can tell, there is nothing like that out-of-the-box in Azure DevOps.

That said, there is an argument that you don't want to check the number of tests but the code that needs to be tested. This argument needs to be stressed.

If you are using YAML, you can add this after your test task is completed:

- task: BuildQualityChecks@9 
  displayName: 'Ensure Code Coverage Quality'
  inputs:

    # the following ensures that code coverage is at least 76% covered
    checkCoverage: true
    coverageFailOption: fixed  
    coverageType: lines # you can update this based on how you run your tests
    coverageThreshold: 76

    # the following ensures there are no build warnings from prior tasks
    checkWarnings: true
    warningFailOption: fixed
    warningThreshold: 0

To get access the [BuildQualityChecks@9] task, you would have to install "Build Quality Checks" from the Azure market place: https://marketplace.visualstudio.com/items?itemName=mspremier.BuildQualityChecks

Answer 2

As far as I know, we cannot set the task itself to make the task/pipeline fail when it cannot find the file.

For a workaround:

You could use the Build Quality Checks task from Build Quality Checks extension.

This task can scan all set tasks and check warnings. If the number of warnings is greater than the set upper limit, the pipeline will fail.

Result: