What strategy to use with xUnit for integration tests when knowing they run in parallel?

Question

I use dotnet core with xUnit for my unit tests and also for my integration tests. I have a base abstract class for all my tests following the Given-Then-When philosophy in this way:

namespace ToolBelt.TestSupport
{
    public abstract class Given_WhenAsync_Then_Test
        : IDisposable
    {
        protected Given_WhenAsync_Then_Test()
        {
            Task.Run(async () => { await SetupAsync();}).GetAwaiter().GetResult();
        }

        private async Task SetupAsync()
        {
            Given();
            await WhenAsync();
        }

        protected abstract void Given();

        protected abstract Task WhenAsync();

        public void Dispose()
        {
            Cleanup();
        }

        protected virtual void Cleanup()
        {
        }
    }
}

As a summary, for each fact (then), the constructor (given) and the action (when) are executed again. This is very interesting to achieve idempotent tests because every fact should be runnable in isolation (the given should be idempotent). This is great for unit tests.

But for integration tests sometimes I find issues in scenarios like this:

I have a mongoDb repository implementation I want to test. I have tests to verify that I can write on it, and others to verify that I can read from it. But since all these tests run in parallel I have to be very mindful of how to setup the Given and how and when to clean the context.

Test class A:

1. Given: I write to database a document
2. When: I read the document
3. Then: the result is the expected document

Test class B:

1. Given: Repo available
2. When: I write a document
3. Then: it writes without exceptions

Now imagine that both test classes are run in parallel, sometimes the following problems arise:

• Test A executes and writes a document with Id 1. At the same time Test B tries to write a document with Id 1 in its when and it fails because there is a document already in the same database with the same Id.
• Test B executes, and it has a teardown/cleanup that deletes the document at the end of its test. At the same time the Test A was about to read a document that expects to be there... and it fails because the document has been deleted (from the Test B)

The question is: Is it even possible to run integration tests in parallel and achieve idempotent Given without facing problems because one test messes up with the data of the other test?

I thought of a few ideas but I have no experience with it, so I am looking for opinions and a solution.

• Solution A: Ensuring every test class uses data that no other test accesses. For example by providing different Ids to the test data. This could indeed solve the problem but it forces developers to be aware of which Ids are being used by other tests.
• Solution B: Have some kind of assembly Given and Teardown that prepares the scenario for every test. Again, we would rely on something greater than the test class itself and it seems it violates the GivenThenWhen philosophy I want to follow.

xUnit has the possibility of different shared context between tests but I don't see how that fits with my template either: https://xunit.github.io/docs/shared-context.

How do you handle those integration test scenarios with xUnit? Ta

---

UPDATE 1: Here is an example of how I create my tests using the GTW philosophy and xUnit. These facts sometimes fail because they cannot insert a document with an Id that already exists (because other test classes that use a document with the same id are running at the same time and didn't cleanup yet)

public static class GetAllTests
{
    public class Given_A_Valid_Filter_When_Getting_All
        : Given_WhenAsync_Then_Test
    {
        private ReadRepository<FakeDocument> _sut;
        private Exception _exception;
        private Expression<Func<FakeDocument, bool>> _filter;
        private IEnumerable<FakeDocument> _result;
        private IEnumerable<FakeDocument> _expectedDocuments;

        protected override void Given()
        {
            _filter = x => true;
            var cursorServiceMock = new Mock<ICursorService<FakeDocument>>();
            var all = Enumerable.Empty<FakeDocument>().ToList();
            cursorServiceMock
                .Setup(x => x.GetList(It.IsAny<IAsyncCursor<FakeDocument>>()))
                .ReturnsAsync(all);
            var cursorService = cursorServiceMock.Object;

            var documentsMock = new Mock<IMongoCollection<FakeDocument>>();
            documentsMock
                .Setup(x => x.FindAsync(It.IsAny<Expression<Func<FakeDocument, bool>>>(),
                    It.IsAny<FindOptions<FakeDocument, FakeDocument>>(), It.IsAny<CancellationToken>()))
                .ReturnsAsync(default(IAsyncCursor<FakeDocument>));
            var documents = documentsMock.Object;

            _sut = new ReadRepository<FakeDocument>(documents, cursorService);
            _expectedDocuments = all;
        }

        protected override async Task WhenAsync()
        {
            try
            {
                _result = await _sut.GetAll(_filter);
            }
            catch (Exception exception)
            {
                _exception = exception;
            }
        }

        [Fact]
        public void Then_It_Should_Execute_Without_Exceptions()
        {
            _exception.Should().BeNull();
        }

        [Fact]
        public void Then_It_Should_Return_The_Expected_Documents()
        {
            _result.Should().AllBeEquivalentTo(_expectedDocuments);
        }
    }

    public class Given_A_Null_Filter_When_Getting_All
        : Given_WhenAsync_Then_Test
    {
        private ReadRepository<FakeDocument> _sut;
        private ArgumentNullException _exception;
        private Expression<Func<FakeDocument, bool>> _filter;

        protected override void Given()
        {
            _filter = default;
            var cursorService = Mock.Of<ICursorService<FakeDocument>>();
            var documents = Mock.Of<IMongoCollection<FakeDocument>>();
            _sut = new ReadRepository<FakeDocument>(documents, cursorService);
        }

        protected override async Task WhenAsync()
        {
            try
            {
                await _sut.GetAll(_filter);
            }
            catch (ArgumentNullException exception)
            {
                _exception = exception;
            }
        }

        [Fact]
        public void Then_It_Should_Throw_A_ArgumentNullException()
        {
            _exception.Should().NotBeNull();
        }
    }
}

---

UPDATE 2: If I make random Ids, sometimes I will also get into problems because the expected documents to be retrieve from DB contain more items than the ones the test expects (because, again, other tests running in parallel have written more documents in the database).

Answer 1

I found out time ago that xUnit has built in support for asynchronous scenarios with IAsyncLifetime.

So now, as per my open source library

I have the following GivenWhenThen template

using System.Threading.Tasks;
using Xunit;

namespace Sasw.TestSupport.XUnit
{
    public abstract class Given_When_Then_Test_Async
        : IAsyncLifetime
    {
        public async Task InitializeAsync()
        {
            await Given();
            await When();
        }

        public async Task DisposeAsync()
        {
            await Cleanup();
        }

        protected virtual Task Cleanup()
        {
            return Task.CompletedTask;
        }

        protected abstract Task Given();

        protected abstract Task When();
    }
}

As per parallel execution approach, I usually create unique database, stream, etc. at the test's preconditions so that I don't have any logical shared resource at all. Same server, different streams, database or collection for each test.

If, for whatever reason, I need to disable parallel xUnit test execution, the first thing I do is re-consider my design approach, maybe it's a smell. If, still, some tests fail when running in parallel because there is some shared resource I don't have much control over, I disable the parallel xUnit behavior by adding on my test project a folder Properties with a file AssemblyInfo where I instruct xUnit that on that project/assembly, tests should run sequentially.

using Xunit;
#if DEBUG
[assembly: CollectionBehavior(DisableTestParallelization = false)]
#else
// Maybe in DEBUG I want to run in parallel and 
// in RELEASE mode I want to run sequentially, so I can have this.
[assembly: CollectionBehavior(DisableTestParallelization = true)]
#endif

---

More info (in Spanish, sorry) about some testing approach that use the above here on one of my video tutorials: https://www.youtube.com/watch?v=dyVEayGwU3I