CODEWITHSUNDEEP
c#nunit
Dave Becker
Dave Becker

Reputation: 1433

NUnit 3 Assert.That throws NullReferenceException when values are not null

This is driving me crazy! I am using NUnit version 3.6.0 and I am getting unexpected results from my tests. I have created a really simple test to demonstrate the problem I have:

[TestFixture]
public class NunitTest
{
    [Test]
    public void TestIt()
    {
        string x = "x";
        string y = "y";

        // this fails (expected) but with NullReferenceException (unexpected)
        Assert.That(x, Is.EqualTo(y));
    }
}

When the test runs I get a NullReferenceException when I am expecting something like "expected is 'X', actual is 'Y'"

I have furthered the asserts and they all pass

[TestFixture]
public class NunitTest
{
    [Test]
    public void TestIt()
    {
        string x = "x";
        string y = "y";

        // this passes
        bool atest = x.Equals(y);
        Assert.IsFalse(atest);

        // this passes
        Assert.IsNotNull(x);
        // this passes
        Assert.IsNotNull(y);

        // this fails (expected) but with NullReferenceException (unexpected)
        Assert.That(x, Is.EqualTo(y));
    }
}

The other weird thing is that when the values are equal the test passed correctly. Also worth noting I get exactly the same problem when I use Assert.AreEqual(x, y)

Stack Trace:

   at NUnit.Framework.Assert.ReportFailure(String message)
   at NUnit.Framework.Assert.ReportFailure(ConstraintResult result, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression)
   at VSI.Tests.NunitTest.TestIt() in c:\Users\DavidBecker\Documents\Visual Studio 2013\Projects\VSI.Tests\VSI.Tests\NunitTest.cs:line 30

Tools:

.NET Framework version 4.5.1
Visual Studio 2013 Professional
Resharper Version 9

EDIT: I am running the tests from Resharper.

Upvotes: 2

Views: 2540

Answers (2)

Chris
Chris

Reputation: 6042

You're right that ReSharper 9 is the problem.

ReSharper didn't support NUnit 3 until ReSharper version 10. There are however are a number of similarities between NUnit versions 2 and 3, which mean that some older NUnit 2 runners will appear to run NUnit 3 tests - however, this won't be done accurately, and is neither supported or advised.

In the final version of NUnit 2 (2.6.4) - an exception was put in, such that a runner would error if attempting to load NUnit 3 tests would fail immediately, instead of attempting to run tests in an unsupported manner. It seems likely however that ReSharper 9 is using a runner older than NUnit 2.6.4, which doesn't throw this error.

The reason this has only become apparent to you with NUnit 3.6 is due to an internal change with how assertions are registered within NUnit. The coincidental previous success now no longer succeeds - although in many ways, this is an advantage, as it gets rid of this right-side failure, where tests appear to be passing, but aren't actually running correctly.

ReSharper supports NUnit 3 from version 10, although that was the point I believe when they switched from a one-off purchase model to a subscription model. As another option - consider the NUnit 3 Test Adapter (NuGet package or VSIX extension) - which allows you to run NUnit tests individually, using Visual Studio's own test window.

(If your interested in further info, this issue came up recently over on GitHub also: https://github.com/nunit/nunit/issues/1992)

Upvotes: 2

Dave Becker
Dave Becker

Reputation: 1433

Running NUnit.3.6.0 tests in Resharper (9) Unit Test Runner has unexpected results. Use NUnit.ConsoleRunner.3.6.0 instead.

(this worked for me, if anyone has better way or can make it work in RS9 then please let me know).

Upvotes: 0

Related Questions