CODEWITHSUNDEEP
asp.netazure-devopswebformscontinuous-integrationazure-pipelines
Nightscape
Nightscape

Reputation: 464

Failing test running successful in Azure DevOps Pipeline

I have a ASP.NET Web Forms project which is hosted on a Azure DevOps Repository. The Source Control is TFS.

I Created a continuous integration Pipeline in Azure Devops for the Repository. Here is a Image of the Pipeline:

enter image description here

All the Task from the Pipeline are running correctly beside the VSTest-TestAssemblies Task. My NUnit Test is failing in my local Visual Studio, as it should:

enter image description here

But when running my CI, it tells my that no test failed: enter image description here

Here you can see my settings and the path to my tests of the VSTest-TestAssemblies Task: enter image description here

This is what the Folder Structure of my Project looks like:

NR.TestAutomation
- NR.TestAutomation
- NR.TestAutomation.Tests
  - bin
  - obj
  - DemoTests.cs
  - NR.TestAutomation.Tests.csproj
- packages
- NR.TestsAutomation.sln

The Output of the CI Run tells me, that the folder exist and it even finds the test:

Source filter: **/*Tests/*.cs
SystemVssConnection exists true

vstest.console.exe "D:\a\9\s\NR.TestAutomation.Tests\DemoTests.cs"

Why are my failing tests not failing inside the Azure DevOps Pipeline?

Upvotes: 0

Views: 1081

Answers (1)

rickvdbosch
rickvdbosch

Reputation: 15621

You should specify the (output of the) TestProject to run, not the individual files that contain a test. The test runner cannot execute a .cs file.
So in short; the failing test doesn't run successful in an Azure DevOps pipeline... it doesn't run in the pipeline at all.

Test files
Run tests from the specified files. Ordered tests and webtests can be run by specifying the .orderedtest and .webtest files respectively. To run .webtest, Visual Studio 2017 Update 4 or higher is needed. The file paths are relative to the search folder. Supports multiple lines of minimatch patterns.

Source Visual Studio Test task - Arguments

Possibly interesting as well: more info on File matching patterns reference

EDIT:

what would be the proper Source Path to my tests?

Of course that depends on the name of your test project. Looking at your screenshots, you could use **/*.Tests.dll. This recursively matches all the assemblies where the name ends with .Tests.dll.
Since your project's name is NR.TestAutomation.Tests, the assembly should be named NR.TestAutomation.Tests.dll so it will match. It's also ready for future additions of test projects, as long as their name ends with .Tests.

Upvotes: 1

Related Questions