CODEWITHSUNDEEP
nunitxunit
SiberianGuy
SiberianGuy

Reputation: 25312

Run long-running NUnit/xUnit tests so that they are not blocking other tests

I'm running a set of integration tests and while most of them finish within reasonable timeline, there're two tests that are waiting for specific conditions (financial markets conditions to be precise) and they can last for 2-3 hours. So ideally I'd like to achieve two things:

  1. Start those two tests after other tests are finished
  2. Run them in parallel

Is there a way to achieve that in NUnit/XUnit (or other test runner)?

Upvotes: 3

Views: 1081

Answers (2)

Michael Ushakov
Michael Ushakov

Reputation: 2789

  1. Parallel test run depends on arguments of test runner, if you are using xUnit console test runner there is a -parallel argument or MSBuild Options, see: https://xunit.net/docs/running-tests-in-parallel. But in any case you have to split your long running tests on separate Test Classes.

  2. It is harder to guarantee order of test running you could use TestCollection (however according to quide collection running sequentially). You could rename your long running test to place them at the end of list, i.e TestClass2 will be executed after TestClass1. You also could use category attribute parameter to separate tests and run them via 2 commands from dotnet test --filter=TestCategory=LongTests (one for long and another for others), see https://learn.microsoft.com/ru-ru/dotnet/core/testing/selective-unit-tests?pivots=mstest

Upvotes: 3

Rahul Iyer
Rahul Iyer

Reputation: 21025

Start those two tests after other tests are finished

You could keep those two tests in a separate nunit test project, allowing you to run all the other tests separately.

For running tests in parallel, this blog has a nice article:

https://blog.sanderaernouts.com/running-unit-tests-in-parallel-with-nunit

  • Mark your test fixtures with the Parallelizable attribute and set the parallel scope to ParallelScope.All.

  • Create a private class called TestScope and implement IDisposable.

  • Put all startup and clean-up logic inside the TestScope constructor and .Dispose() method respectively.

  • Wrap your test code in a using (var scope = new TestScope) { ... } block

[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {

    [Test]
    public void MyParallelTest() {
        using(var scope = new TestScope()) {
            scope.Sut.DoSomething();
            scope.Repository.Received(1).Save();
        }
    }

    private sealed class TestScope : IDisposable {
        public IRepository Repository{get;}
        public MyClass Sut {get;}
        public TestScope() {
            Repository = Substitute.For<IRepository>();
            Sut = new MyClass(Repository);
        }

        public void Dispose() {
            //clean-up code goes here
            Repository?.Dispose()
        }
    }
}

You should take precautions to ensure that while running in parallel, your tests do not interfere with each other.

As the article states:

How to safely run tests in parallel

To allow tests to run in parallel without them interfering with each other, I have been applying the following pattern for a while:

  • Create a nested private TestScope class that implements IDisposable.
  • All initialization or startup code that would go into the SetUp method goes into the constructor of the TestScope class.
  • Any clean-up or teardown code that would go into the TearDown method goes into the Dispose method All tests run inside a using block that handles the creation and disposal of the TestScope.
[TestFixture]
[Parallelizable(ParallelScope.All)]
public class MyClassTests {

    [Test]
    public void MyParallelTest() {
        using(var scope = new TestScope()) {
            scope.Sut.DoSomething();
            scope.Repository.Received(1).Save();
        }
    }

    private sealed class TestScope : IDisposable {
        public IRepository Repository{get;}
        public MyClass Sut {get;}
        public TestScope() {
            Repository = Substitute.For<IRepository>();
            Sut = new MyClass(Repository);
        }

        public void Dispose() {
            //clean-up code goes here
            Repository?.Dispose()
        }
    }
}

The article provides more valuable advice. I suggest reading it, and thanking the author.

Upvotes: 3

Related Questions