Run NUnit tests in Azure DevOps pipeline

Question

Is it possible? (It seems not. Does it not support NUnit? What should I use instead?)

Here is my test project.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <IsTestProject>true</IsTestProject>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\TheProjectToBeTested\TheProjectToBeTested.csproj" />
  </ItemGroup>

</Project>

And here is my azure-pipelines.yml

pool: 'TFSBuild'

variables:
# The web app will not build because it is netcore3.1 and the server only supports netcore3.0.
  solution: '**/*Build.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: NuGetToolInstaller@0

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

# I added this and I don't undertand it.
- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
     **\$(BuildConfiguration)\*test*.dll
     **\$(BuildConfiguration)\**\*test*.dll
     !**\*Microsoft.VisualStudio.TestPlatform*
     !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'

However the tests don't get run.

##[section]Starting: VSTest
==============================================================================
Task         : Visual Studio Test
Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version      : 2.153.9
Author       : Microsoft Corporation
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
SystemVssConnection exists true
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : E:\TFS\8\s
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
SystemVssConnection exists true
Run the tests locally using vstest.console.exe
========================================================
Test selector : Test assemblies
Test assemblies : **\Release\*test*.dll,**\Release\**\*test*.dll,!**\*Microsoft.VisualStudio.TestPlatform*,!**\obj\**
Test filter criteria : null
Search folder : E:\TFS\8\s
Run settings file : E:\TFS\8\s
Run in parallel : false
Run in isolation : false
Path to custom adapters : null
Other console options : null
Code coverage enabled : false
Diagnostics enabled : false
Rerun failed tests: false
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation.
========================================================
======================================================
[command]"E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" @E:\TFS\_temp\126341f1-33c9-11eb-a545-b3a68600e11b.txt
Microsoft (R) Test Execution Command Line Tool Version 16.3.0-preview-20190715-02
Copyright (c) Microsoft Corporation.  All rights reserved.

vstest.console.exe 
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll"
"E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll"
/logger:"trx"
/TestAdapterPath:"E:\TFS\8\s"
Starting test execution, please wait...
Test run will use DLL(s) built for framework .NETFramework,Version=v4.0 and platform X86. Following DLL(s) do not match framework/platform settings.
TheProjectToBeTested.Test.dll is built for Framework .NETCoreApp,Version=v3.0 and Platform AnyCPU.
Microsoft.TestPlatform.CommunicationUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CoreUtilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.CrossPlatEngine.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
Microsoft.TestPlatform.PlatformAbstractions.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Microsoft.TestPlatform.Utilities.dll is built for Framework .NETStandard,Version=v2.0 and Platform AnyCPU.
NUnit3.TestAdapter.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
testhost.dll is built for Framework .NETCoreApp,Version=v2.1 and Platform AnyCPU.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.

No test is available in E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\TheProjectToBeTested.Test.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CommunicationUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CoreUtilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.CrossPlatEngine.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.PlatformAbstractions.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\Microsoft.TestPlatform.Utilities.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\NUnit3.TestAdapter.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\cs\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\de\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\es\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\fr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\it\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ja\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ko\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pl\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\pt-BR\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\ru\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\testhost.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\tr\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hans\Microsoft.TestPlatform.CrossPlatEngine.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CommunicationUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CoreUtilities.resources.dll E:\TFS\8\s\TheProjectToBeTested.Test\bin\Release\netcoreapp3.0\zh-Hant\Microsoft.TestPlatform.CrossPlatEngine.resources.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Results File: E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx

No Result Found to Publish 'E:\TFS\8\s\TestResults\TFSBUILD$_TFSBUILD_2020-12-01_11_33_48.trx'.
##[section]Async Command Start: Publish test results
##[section]Async Command End: Publish test results
##[section]Finishing: VSTest

Answer 1

If you have a .net core application as you do, the other answer using DotNetCoreCLI@2 with test command is the correct answer.

However, if you are using framework and don't want to use VSTest or really have to use nunit there is another way to get this to work.

You can use the pipeline with a docker container that has nunit in. You can use the code from here which is a mono image that has nunit added. A requirement of Azure Devops is there is no entry point so you will need to use your own copy without the last line.

Once you have your container you can then run your pipeline in a container and use a bash command block

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: '/nunit/nunit3-console.exe **/*Tests/*.csproj --configuration $(buildConfiguration)'

Answer 2

You maybe missing NUnit3TestAdapter package in one of project

<?xml version="1.0" encoding="utf-8"?>
<packages>
   <package id="NUnit" version="3.10.1" targetFramework="net45" />
   <package id="NUnit3TestAdapter" version="4.1.0" targetFramework="net45" />
</packages>

After adding package this task was running all nunit tests

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

Answer 3

Please move rather to dotnet command and instead of using VSTest run it using this:

If you have test projects in your repository, then use the .NET Core task to run unit tests by using testing frameworks like MSTest, xUnit, and NUnit.

steps:
# ...
# do this after other tasks such as building
- task: DotNetCoreCLI@2
  inputs:
    command: test
    projects: '**/*Tests/*.csproj'
    arguments: '--configuration $(buildConfiguration)'

Answer 4

Run NUnit tests in Azure DevOps pipeline

To test .Net Core dlls with Visual Studio Test task, please try to add following additional settings of Visual Studio Test task:

• Path to vstest.console.exe: E:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform.

• Other console options: /framework:".NETCoreApp,Version=v3.0":