CODEWITHSUNDEEP
c#azure-devopsazure-pipelinestest-suitetest-plan
Artsiom
Artsiom

Reputation: 137

How to select a tests form a test suite

I have a stage with the follow tasks: enter image description here

There are 2 test suites with 8 test cases. I want to execute only test cases with tag "download" in feature file enter image description here

When I locally execute command

dotnet test <name>.dll --filter TestCategory=download

everything is fine, but when I try to add command-line options in "Other console options" in task enter image description here test cases are not filtered and all are executed.

What am I doing wrong and what can help me to filter tests?

Upvotes: 1

Views: 720

Answers (2)

Kevin Lu-MSFT
Kevin Lu-MSFT

Reputation: 35194

Matt is correct. The other console options in Visual Studio test task doesn't support running from a test plan.

To solve this issue, you could use .runsettings file.

Now, versions after VS 16.6 preview 3 can support adding testcasefilter to .runsettings directly.

Here is an example:

.runsettings file

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations that affect the Test Framework -->
  <RunConfiguration>
    <MaxCpuCount>1</MaxCpuCount>
    <!-- Path relative to directory that contains .runsettings file-->
    <ResultsDirectory>.\TestResults</ResultsDirectory>
    <TestCaseFilter>TestCategory=xxx</TestCaseFilter>
  </RunConfiguration>
....

Pipeline Settings:

enter image description here

You could install the VS 16.7.1 in Visual Studio Test Platform Installer task.

Then you could set the Test Platform version and runsettings file in the visual studio test task.

enter image description here

Here is a ticket about testcasefilter in runsettings file.

Upvotes: 1

Matt
Matt

Reputation: 4045

The other console options are not honored when running from a test plan. You either need to change it to select tests using an assembly (which matches your local copy) or you need to filter the tests in the suite using a filter criteria not tied in source code.

If you expand the info of that option:

These options are not supported and will be ignored when running tests using the ‘Multi agent’ parallel setting of an agent job or when running tests using ‘Test plan’ option. The options can be specified using a settings file instead.

Upvotes: 1

Related Questions