Azure DevOps - Builds not showing code coverage when using "dotnet test --collect:'Code Coverage'"

Question

I have a .NET Framework project that is being built on a self hosted Windows build agent.

There is a step to run tests, and that step needs to provide code coverage reports and stats.

When i try using "dotnet test" the step runs and the tests complete, the .coverage files are also generated. When i check the build summary after it's complete i see the standard test results and report, and also the code coverage tab. The code coverage tab has a download link to get the file. There is no code coverage report. There is also a link to "Setup Code Coverage" on the initial build summary screen.

Why is there no code coverage report? and why is the "Setup Code Coverage" link still visible?

This is incredibly frustrating! I must be missing something incredibly obvious, but the docs suggest what i have done is correct.

Using VSTest task rather that dotnet tests results in the same outcome, but runs far slower.

        displayName: dotnet test
        inputs: 
          command: test 
          arguments: '--configuration $(BuildConfiguration) --collect:"Code Coverage"'
          workingDirectory: '$(Build.SourcesDirectory)\src'```

Answer 1

I eventually achieved this by using Hugh Lin's answer for help and modifying for my own purposes.

We have Coverlet as a reference in the project, and ReportGenerator installed into Azure DevOps, so that made this a little easier.

I found that we had an issue with a SOAP API reference that was causing the huge performance issues with generating a report. Once I filtered that out with a "classfilter"the process became more manageable. I also found that without the "disable.coverage.autogenerate" variable the "PublishCodeCoverageResults" task will take forever and likely fail as it tries to do the "ReportGenerator" step itself by without the "classfilters". It does this because the "ReportGenerator" is built into the "PublishCodeCoverageResults" step now, but due to having no filters it doesn't work for this scenario.

This is running against a .NET Framework project so there were a few adjustments to the projects needed to ensure "dotnet test" works successfully.

variables:
  disable.coverage.autogenerate: 'true'


  - task: DotNetCoreCLI@2
    displayName: dotnet test
    inputs: 
      command: test 
      publishTestResults: true
      arguments: '/p:CollectCoverage=true /p:CoverletOutputFormat=cobertura --no-restore'
      workingDirectory: '$(Build.SourcesDirectory)\src'
      configuration: "$(buildConfiguration)"
      
  - task: reportgenerator@4
    inputs:
      reports: '$(Build.SourcesDirectory)\src\*.UnitTests\coverage.cobertura.xml'
      targetdir: '$(Common.TestResultsDirectory)/CoverageReport/'
      classfilters: '-NAMESPACE*'

  - task: PublishCodeCoverageResults@1
    inputs:
      codeCoverageTool: 'Cobertura'
      summaryFileLocation: '$(Build.SourcesDirectory)\src\*.UnitTests\coverage.cobertura.xml'
      reportDirectory: '$(Common.TestResultsDirectory)/CoverageReport/'

Answer 2

By default, the code coverage for the dotnet test task is output to a .codecoverage file, which Azure DevOps does not know how to interpret and only provides as a downloadable file. Code coverage Tab only supports code coverage data in Jacoco or Cobertura formats. So the result of the *.coverage file can not be shown by tables and graphs.

If you want more detailed code coverage report, you need to use coverlet in .Net framework by install the tool during the pipeline and then generate the report. For example in PowerShell script:

dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12
dotnet tool install coverlet.console --tool-path . --version 1.4.1
mkdir .\reports
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*test*.dll" }
$coverlet = "$pwd\coverlet.exe"
& $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml"} |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }

Then add Publish code coverage task:

For details, you can refer to the case mentioned in the comment and this ticket.