Azure DevOps 2019 Server failed on executing tests with code coverage

Question

Current situation:

To enable our Azure DevOps 2019 Server (Agents) to build ASP.NET Core 3 and .Net Core 3 applications we installed Build Tools for Visual Studio 2019 (expander Tools for Visual Studio 2019) on it. It's not best practice, but Azure DevOps 2019 Server and Agents are installed on same machine.

Except collecting coder coverage everything works like expected. Applications were build and artifacts were provided. We could confirm, all artifacts after release are working like expected.
Like already mention, everything is fine except code coverage.

The application to build is a .NET Framework 4.6.2 application and was build before VS 2019 build tools installation with VS 2017 Enterprise on the agent successfully.

For testing we are using MSTest with Visual Studio Test in build pipeline, after Visual Studio build task. The test task is configured like followed:

variables:
  SolutionRootDirectory: 'MySolutionRoot'
  SettingsFilePath: ''
  BuildPlatform: 'Any CPU'
  BuildConfiguration: 'Release'

steps:
- task: VSTest@2
  displayName: 'Test Assemblies: $(BuildConfiguration)'
  inputs:
    testAssemblyVer2: |
     **/*Test?(s).dll
     !**/obj/**
    searchFolder: '$(SolutionRootDirectory)'
    runSettingsFile: '$(SettingsFilePath)'
    codeCoverageEnabled: true
    testRunTitle: 'Test run: $(SolutionRootDirectory)\*.sln'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'
    diagnosticsEnabled: true
    collectDumpOn: always
    rerunFailedTests: false

All tests were executed successfully, but the task failed with following message:

##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
##[error]Error: The process 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\Extensions\TestPlatform\vstest.console.exe' failed with exit code 1
Publishing test results to test run '11901'
Test results remaining: 260. Test run id: 11901
##[error]VsTest task failed.

After checking the whole log I found following messages:

Data collector 'Code Coverage' message: Data collector 'Code Coverage' failed to provide initialization information. Error: System.TypeInitializationException: The type initializer for 'Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop' threw an exception. ---> Microsoft.VisualStudio.Diagnostics.Common.InvariantException: Failed to load IntelliTrace Profiler binary or failed to locate functions.

...and a few lines below:

 ---> System.ComponentModel.Win32Exception: The system cannot find the path specified
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Diagnostics.Common.Check.Throw[XT](String message, Func`1 innerEx)
   at Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop.ThrowInvariantExceptionOnZeroPtr(IntPtr ptr, String message)
   at Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop.InitInterop()
   at Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop..cctor()
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop.get_InteropInterface()
   at Microsoft.VisualStudio.Diagnostics.Logging.LoggingConfig.Publish()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.InitiateCollection()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectorInformation.SetTestExecutionEnvironmentVariables()
   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.GetEnvironmentVariables(Boolean& unloadedAnyCollector).

Question

Does anybody have an idea what the issue could be?
I guess, maybe we have to install Agents for Visual Studio 2019 (expander Tools for Visual Studio 2019), but I'm not sure if it is the correct approach. Furthermore, I could not find any meaningfully information which installation packe we have to use - Agent or Controller.

In the past, most of the time, a fully Visual Studio Enterprise installation was done, to achieve a completely feature set for agents. But I want a more clean approach and want to install only needed packages and to avoid to use an Enterprise licence for a build agent.

Ideas, approaches and best practices are very welcome. Thank you very much for any help.

Answer 1

Using Visual Studio Test Platform Installer task worked for me.

Just make sure you change vsTestVersion to toolsInstaller

- task: VisualStudioTestPlatformInstaller@1
  inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'specificVersion'
    testPlatformVersion: '16.11.0'

- task: VSTest@2
  inputs:
    ...
    vsTestVersion: 'toolsInstaller'
    codeCoverageEnabled: true

(tested with version 16.11.0, released 3h ago, not sure if it was available previously)

There is one downside for me, previously I could use one variable/parameter to set both VSTest and VSBuild version as both have version parameters using same format (15.0, 16.0, latest), but the tool installer requires a specific version, so it's not straightforward how to keep that synced, however it is also not clear if we need to, perhaps not.

Answer 2

applied solution

Well, after a lot of research and reading about best practices, the most practicable and most target oriented solution was to install a Visual Studio 2019 Enterprise on the build agent. It was my first approach, but asked the question for finding another solution, to avoid installing a whole VS 2019 Enterprise on the build agent.

It contains all needed assemblies and code coverage is part of Enterprise edition only.

I preferred that solution because of no build pipeline modifications are needed. Otherwise all existing build pipelines must be modified to match other suggested solutions.
If it's acceptable to edit existing pipelines, following are listed other suggested solutions.

all suggested solutions:

1. using Coverlet (Coverlet.MsBuild) by installing it via NuGet into test projects and importing its results (maybe in combination use with ReportGenerator
2. installing Test Agent/Controller and referencing its vstest.console.exe in build pipeline's MSTest task
3. copy needed binaries to relevant directories on build agent

Thank you for suggestions and your invested time.