CODEWITHSUNDEEP
c#unit-testingnunitasync-await
Peter
Peter

Reputation: 1715

Async unit tests not working as expected

I am using the latest version of NUnit (2.6.2) in Visual Studio 2012 using both resharper and the visual studio test runner. I have the follow sample tests in which I am attempting to verify that an exception is raised on an expected asynchronous method call.

Unfortuantely, this doesn't seem to be working as expected. The first test AsyncTaskCanceledSemiWorking only works because i have the expectedexception attribute. The actual assert is completely ignored (as you can see by the ArgumentOutOfRange exception which is just a fake to get it to fail).

The AsyncTaskCanceledWorking works fine, but doesnt test that the exception is thrown on a specified line, hence less useful.

The third fails majestically with the below....

System.Threading.Tasks.TaskCanceledException : A task was canceled.
Exception doesn't have a stacktrace

Any ideas on how I can test for the TaskCanceledException from a specific line would be very useful.

Thanks

    [Test]
    [ExpectedException(typeof(TaskCanceledException))]
    public async Task AsyncTaskCanceledSemiWorking()
    {
        CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
        CancellationToken token = cancellationTokenSource.Token;

        cancellationTokenSource.Cancel();

        Assert.That(await LongRunningFunction(token), Throws.InstanceOf<ArgumentOutOfRangeException>());


    }

    [Test]
    [ExpectedException(typeof(TaskCanceledException))]
    public async Task AsyncTaskCanceledWorking()
    {
        CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
        CancellationToken token = cancellationTokenSource.Token;

        cancellationTokenSource.Cancel();

        int i = await LongRunningFunction(token);
    } 


    [Test]
    public async Task AsyncTaskCanceledFailed()
    {
        CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
        CancellationToken token = cancellationTokenSource.Token;

        cancellationTokenSource.Cancel();

        Assert.That(await LongRunningFunction(token), Throws.InstanceOf<TaskCanceledException>());

    }

    public async Task<int> LongRunningFunction(CancellationToken token)
    {
        token.ThrowIfCancellationRequested();

        await Task.Delay(1000, token);

        return 5;
    }

Upvotes: 2

Views: 7987

Answers (3)

Anthony
Anthony

Reputation: 5226

If you use Resharper 7.1 or later with NUnit 2.6.2 or later, Test methods that are public async void will work. Resharper 7.1 was released today (13 November 2012)

Earlier versions of the Resharper test runner do not wait for the test to complete and can pass without actually testing anything.

Versions of NUnit prior to 2.6.2 had much the same problem.

The Resharper NUnit test runner is not the same codebase as the NUnit GUI and commandline test runners, and is updated separately by Jetbrains.

Upvotes: 0

Ravichandran Jv
Ravichandran Jv

Reputation: 1

Peter Unfortuantely, this doesn't seem to be working as expected. The first test AsyncTaskCanceledSemiWorking only works because i have the expectedexception attribute. The actual assert is completely ignored (as you can see by the ArgumentOutOfRange exception which is just a fake to get it to fail).

The expectation is wrong because the attribute takes precedence over the Assert.Throws.

As Simone has demonstrated, the expectation is to "assert" that the exception is an InstanceOf and therefore, it belongs to Assertion and not to throws. I think your expectation is probably based on the way Throws is used in Java.

Upvotes: -2

Simone
Simone

Reputation: 3996

I assume you want to check that LongRunningFunction will throw a TaskCanceledException.

I think that the behavior you are experiencing is entirely correct, and the misunderstanding is in this statement:

await LongRunningFunction(token)

Here you are effectively executing the asynchronous operation and waiting for it to complete, which will also rethrow the first exception that occurred in its invocation. You could basically replace that with:

throw new TaskCanceledException()

Hence why the first two tests are succeeding - you are using the ExpectedExceptionAttribute - and the third failing - you're not expecting the exception.

The first argument to Assert.That, when you are later using Throws, should be a delegate of some sort, as NUnit would have to invoke it in order to catch the exception that bubbles up from its invocation. If you invoke it yourself there's of course no way for NUnit to trap the exception besides using the ExpectedExceptionAttribute.

In other words, the ideally correct way to do it is:

// WARNING: this code does not work in NUnit <= 2.6.2
Assert.That(async () => await LongRunningFunction(token), Throws.InstanceOf<TaskCanceledException>());

I would like to tell you that NUnit supports this syntax for asynchronous methods, which would be very natural and allow you to test for the exception in a specific part of the code, but it doesn't and the test would fail reporting that you were expecting an exception but no exception occurred.

The reason is that in order to get the exception from the invocation of that asynchronous anonymous method NUnit would have to await it, which it currently doesn't.

One alternative that I could give you is to use a non-async lambda where you Wait on the task returned by the async operation, but the syntax is unfortunately not as nice because awaiting an async operation behaves in a different way from waiting on the task it returns. Specifically, in the case of an exception thrown from the async operation, you would get the actual exception in the first case and an AggregateException in the second. In any case here's some code which would work with 2.6.2:

var aggregate = Assert.Throws<AggregateException>(() => LongRunningFunction(token).Wait());
Assert.IsInstanceOf<TaskCanceledException>(aggregate.InnerExceptions.Single());

To conclude, although NUnit 2.6.2 indeed introduced support for async test methods, which allows you to write async [void|Task|Task<T>] tests, we didn't consider to extend the support to async anonymous methods which would come useful in this kind of assertions, although I believe we could.

Upvotes: 10

Related Questions