CODEWITHSUNDEEP
c#.net-corenunitdotnet-cli
Dejan
Dejan

Reputation: 10373

How to filter NUnit tests by category using "dotnet test"

I have a project that has a

[TestFixture, Category("Oracle")]

and a

[TestFixture, Category("OracleOdbc")]

with a couple of tests which I would like to execute separately using dotnet test.

Here's what I tried after some Googling:

  1. dotnet test MyProject.csproj --where "cat==Oracle" but this switch does not exists anymore.
  2. dotnet test MyProject.csproj --filter Category="Oracle" yields 0 applicable tests: No test is available in ....

Then, I've stumbled over this article and although it describes MSTest (and NUnit has the CategoryAttribute and not a TestCategoryAttribute), I've tried

  1. dotnet test MyProject.csproj --filter TestCategory="Oracle"

Bingo. This time all "Oracle" tests were executed. But now comes the confusing part. If I run dotnet test MyProject.csproj --filter TestCategory="OracleOdbc", all tests are being executed, including "Oracle" and "OracleOdbc". This makes me wonder if TestCategroy is the proper way to go for NUnit or if this is a bug.

I'm using .NET Command Line Tools (2.1.2) and the project references are:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
<PackageReference Include="NUnit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
<PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.7" />

BTW, I don't know if it matters but my test project is multi-targeting netcoreapp2.0 and net462.

Upvotes: 26

Views: 23658

Answers (5)

Alexander Petrovskiy
Alexander Petrovskiy

Reputation: 309

Try using the pipe character for union of two sets of tests:

dotnet test --no-build --verbosity normal --filter Category="Oracle|OracleOdbc"

Alternatively, you can use the Test Selection Language:

dotnet test --no-build --verbosity normal -- NUnit.Where="cat == Oracle && cat == OracleOdbc"

Used with .NET 8 (I believe and the previous ones):

    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0" />
    <PackageReference Include="NUnit" Version="4.1.0" />
    <PackageReference Include="NUnit.Allure" Version="1.2.1.1" />
    <PackageReference Include="NUnit.Analyzers" Version="4.2.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />

and installation like that:

# testing
dotnet add "${TEST_PRJ_FOLDER}" package Microsoft.NET.Test.Sdk --prerelease
dotnet add "${TEST_PRJ_FOLDER}" package NUnit --prerelease
dotnet add "${TEST_PRJ_FOLDER}" package NUnit3TestAdapter --prerelease
dotnet add "${TEST_PRJ_FOLDER}" package NUnit.Allure
dotnet add "${TEST_PRJ_FOLDER}" package Allure.Commons --prerelease
dotnet add "${TEST_PRJ_FOLDER}" package NUnit.Analyzers --prerelease

Upvotes: 1

cbp
cbp

Reputation: 25638

There are now two options for filtering tests by category using dotnet test. You can use either dotnet.exe's built-in test filtering syntax, or the NUnit filtering syntax.

First add the NUnit3TestAdapter to your project using NuGet:

install-package nunit3testadapter -proj YourProject

You can then filter tests either like this:

dotnet.exe test .\ --test-adapter-path:. --filter TestCategory=Foo

or like this:

dotnet.exe test .\ --test-adapter-path:. -- NUnit.Where="cat=Foo"

This blog post goes into further detail.

Upvotes: 1

M.Hassan
M.Hassan

Reputation: 11072

In Nunit Framework, Category Attribute can be on the level of the Method.

Example:

public class NUnitTest 
    {
        [Test]
        [Category("CategoryA")] 
        public void TestMethod1()
        {
        }

        [Test]
        [Category("CategoryB")] 
        public void TestMethod2()
        {
        }

    }

and command is:

dotnet test --filter TestCategory=CategoryA     #Runs tests which are annotated with [Category("CategoryA")].

Also, there are many options on the level of method and others For more details: read

Upvotes: 4

Tono Nam
Tono Nam

Reputation: 36080

You can also place a filter inside your test if you like enums like me:

    [Test]
    public void Test_Foo()
    {            
        // filter test
        switch (GlobalDefinitions.Category)
        {
                // OK Test
                case Category.Oracle:
                case Category.SQL:
                    break;

                // Do NOT test
                case Category.MongoDb:
                    Assert.Inconclusive();

                // Error
                default:
                    Assert.Fail("Not implemented case");
        }

        // perform test...

    }

Have the variable GlobalDefinitions.Category obtain the value from a resource file or whatever works best for you.

Edit make the same code shorter with Flags

Create your categories

[Flags] // <-------------------- Important to shorten code 
public enum Category: int
{
    None = 0,
    Oracle = 1 << 0,
    SQL = 1 << 1,
    MongoDb = 1 << 2,

    // future categories        

    ALL = -1
}

// create your filter method

public static void Filter(Category category)
{
    if(GlobalDefinitions.Category.HasFlag(category) == false)
       Assert.Inconclusive(); // do not perform test

    // else perform test                         
}

// then create your test as:

[Test]
public void Test_Foo()
{ 
    Filter(Category.SQL | Category.MongoDb); // place filter (in this test we are testing for SQL and MongoDb

    // perform your test ...

}

Upvotes: -2

Douglas Waugh
Douglas Waugh

Reputation: 328

This might not be very helpful, but it seems to be working for me correctly. I created the projects using the dotnet-cli.

First I installed the NUnit3 test adapter instructions from here. This only needs to be run once on each machine so you don't need to do it again if you have already run it.

dotnet new -i NUnit3.DotNetNew.Template

Then I created my solution, created my test project and added the test project to the solution.

dotnet new sln -n Solution
dotnet new nunit -n TestProject -o tests\TestProject
dotnet sln add tests\TestProject\TestProject.csproj

Then I updated UnitTest1.cs to include two test fixtures, one with the category Oracle and one with the category OracleOdbc.

using NUnit.Framework;

namespace Tests
{
    [TestFixture]
    [Category("Oracle")]
    public class OracleTests
    {
        [Test]
        public void OracleTest()
        {
            Assert.Fail();
        }
    }

    [TestFixture]
    [Category("OracleOdbc")]
    public class OracleOdbcTests
    {
        [Test]
        public void OracleOdbcTest()
        {
            Assert.Fail();
        }
    }
}

Then I can specify which category I choose to run.

dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="Oracle"

or

dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="OracleOdbc"

both run only one test and the message shows it is the correct test that fails.

Using DotNet-Cli version 2.1.4 and NUnit3TestAdapter version 3.9.0

Upvotes: 19

Related Questions