How to ensure a Debug.Assert fires correctly using NUnit

Question

I've been trying to wrap my head around this issue: How to create a unit test that tests if a function aborts because a Debug.Assert (from System.Diagnostics) fails, marking it as passed when it does this and as failed if it doesn't.

I knowNUnit has the feature [ExpectedException(typeof( ArgumentException ) )], but I cannot seem to find out what kind of exception it is from the MSDN website. Intuition would say it might be something like an AssertionException, and that one does exist... but is part of the NUnit framework. I guess that's the exception NUnit assert throw around. I might be able to just nuke it by using:

[ExpectedException(typeof(Exception))]

But this gives rise to the problem that the standard windows debugging window shows up. In my search I came across ways to remove this window from showing up at all, but this feels like taking a butcheringknife to the surgerytable where you normally use a scalpel. Because I do want to be able to see this window when something unexpected happens, when I would execute my program.

I guess there is the workaround of replacing the Debug.Assert method with the NUnit counterpart (I'm still early in my project, so it's not a too big of a refactoring), but I assume a lot of programmers stick with Debug.Assert functionality as it's standard in .NET.

As such, I'd like to know how to 'assert' Debug.Assertion failures, without having to 'murder' the Windows debugging screen from my project?

To have an concrete example from a contract in my code, there is a sample below. For those it might seem familiar, it is the To-Wound table from Warhammer 40K tabletop wargame written as a function.

static public int assaultToHit(int _attacker_WeaponSkill,
            int _defender_WeaponSkill)
        {
            //Preconditions
            Debug.Assert(_attacker_WeaponSkill >= 1 && _attacker_WeaponSkill <= 10,
                "Weapon Skill stat must be in range [1,10]");
            Debug.Assert(_defender_WeaponSkill >= 1 && _defender_WeaponSkill <= 10,
                "Weapon Skill stat must be in range [1,10]");

            int target;
            if (_attacker_WeaponSkill > _defender_WeaponSkill)
            {
                target=3;
            }
            else if (_defender_WeaponSkill >= (_attacker_WeaponSkill + _attacker_WeaponSkill + 1))
            {
                target=5;
            }
            else
            {
                target=4;
            }

            //postconditions
            Debug.Assert(target >= 3 && target <= 5,
                "To hit target for assault must be in range [3,5]");

            return target;
        }

The function to test the preconditions would be in the line of something like this:

    [TestCase(-1,2)]
    [TestCase(1, -2)]
    [TestCase(-1, -2)]
    [TestCase(11, 2)]
    [TestCase(1, 20)]
    [TestCase(11, 20)] 
    [ExpectedException(typeof(Exception))]
    public void testContract_AssaultToHit(int _attacker_weaponskill, 
        int _defender_weaponskill)
    {
        Warhammer40kRules.assaultToHit(_attacker_weaponskill, 
            _defender_weaponskill);
    }

Answer 1

From https://stackoverflow.com/a/117247/605538 one can find a recommendation to use exceptions for public interfacing, while using assertions to verify internal code. When unit-testing preconditions of a function (and a similar thought can be applied for post-conditions, altough I personally would prefer to use assertions there), one can therefore be recommended to use exceptions instead of using assertions.

Using exceptions instead of would result in a sample like this:

static public int assaultToHit(int _attacker_WeaponSkill,
        int _defender_WeaponSkill)
    {
        //Preconditions
        if(!(_attacker_WeaponSkill >= 1 && _attacker_WeaponSkill <= 10))
        {
            throw new ArgumentOutOfRangeException("Attackers WeaponSkill must be in range [1,10]");
        }
        if(!(_defender_WeaponSkill >= 1 && _defender_WeaponSkill <= 10))
        {
            throw new ArgumentOutOfRangeException("Defenders WeaponSkill must be in range [1,10]");
        }

        ...
        //rest unchanged
    }

Coupled with the following NUnit test:

[TestCase(-1,2)]
[TestCase(1, -2)]
[TestCase(-1, -2)]
[TestCase(11, 2)]
[TestCase(1, 20)]
[TestCase(11, 20)] 
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void testContract_AssaultToHit(int _attacker_weaponskill, 
    int _defender_weaponskill)
{
    Warhammer40kRules.assaultToHit(_attacker_weaponskill, 
        _defender_weaponskill);
}

Sidebar [12 June 2014]: I've been recently coming to the opinion that you should not have to test precondition violations in the context of Design by Contract. If you design with DbC in mind, you basically state the following: "If you call a method and meet its preconditions, you are guaranteed a certain behavior. If you do not meet the method's preconditions, you can expect undefined behavior.". In the broadest sense of the term 'undefined behavior', this means that you cannot expect any sort of state. You might get exceptions, maybe nothing happens at all and maybe your hard disk gets formatted (well... you get the point ;) ). As such, you cannot test for 'undefined behavior'.

What you can test, is your defensive programming that makes sure that a precondition failure does not cause the function to operate, for example by throwing an Exception. This behavior is testable.

Answer 2

For your particular case I would recommend you to have a look into Microsoft Code Contracts, as the specific assert case you have is targeted at checking input contracts for your functions.

If Code Contracts is too big for you, I would recommend that you make your contracts throw exceptions when implicit contracts are not adhered to, instead of asserting.

However Debug.Asserts are still valuable, but should be used in situations that are less-likely to happen, such as after a statement that makes a call returning null instead of the empty collection that you expect.

Answer 3

It seems you have used wrong facilities to control an application flow in case of an error. Debug.Assert() should not be used to drive an application logic flow.

Unit Test should cover a real test case and it seems either you're trying to implement wrong test case or you need to throw an exception/etc rather than using Debug.Assert(). You can share some code so it would be possible give you some specific advices.

Anyway you can read MSDN about how-to add a custom trace listener and intercept Assert calls.

Useful links:

• Debug.Assert Method
• Walkthrough: Creating a Custom Trace Listener