How do you unit-test a method with complex input-output

Question

When you have a simple method, like for example sum(int x, int y), it is easy to write unit tests. You can check that method will sum correctly two sample integers, for example 2 + 3 should return 5, then you will check the same for some "extraordinary" numbers, for example negative values and zero. Each of these should be separate unit test, as a single unit test should contain single assert.

What do you do when you have a complex input-output? Take a Xml parser for example. You can have a single method parse(String xml) that receives the String and returns a Dom object. You can write separate tests that will check that certain text node is parsed correctly, that attributes are parsed OK, that child node belongs to parent etc. For all these I can write a simple input, for example

<root><child/></root>

that will be used to check parent-child relationships between nodes and so on for the rest of expectations.

Now, take a look at follwing Xml:

<root>
  <child1 attribute11="attribute 11 value" attribute12="attribute 12 value">Text 1</child1>
  <child2 attribute21="attribute 21 value" attribute22="attribute 22 value">Text 2</child2>
</root>

In order to check that method worked correctly, I need to check many complex conditions, like that attribute11 and attribute12 belong to element1, that Text 1 belongs to child1 etc. I do not want to put more than one assert in my unit-test. How can I accomplish that?

Answer 1

All you need - is to check one aspect of the SUT (System Under Test) in separate test.

[TestFixture]
    public class XmlParserTest
    {
        [Test, ExpectedException(typeof(XmlException))]
        public void FailIfXmlIsNotWellFormed()
        {
            Parse("<doc>");
        }

        [Test]
        public void ParseShortTag()
        {
            var doc = Parse("<doc/>");

            Assert.That(doc.DocumentElement.Name, Is.EqualTo("doc"));
        }

        [Test]
        public void ParseFullTag()
        {
            var doc = Parse("<doc></doc>");

            Assert.That(doc.DocumentElement.Name, Is.EqualTo("doc"));
        }

        [Test]
        public void ParseInnerText()
        {
            var doc = Parse("<doc>Text 1</doc>");

            Assert.That(doc.DocumentElement.InnerText, Is.EqualTo("Text 1"));
        }

        [Test]
        public void AttributesAreEmptyifThereAreNoAttributes()
        {
            var doc = Parse("<doc></doc>");

            Assert.That(doc.DocumentElement.Attributes, Has.Count(0));
        }

        [Test]
        public void ParseAttribute()
        {
            var doc = Parse("<doc attribute11='attribute 11 value'></doc>");

            Assert.That(doc.DocumentElement.Attributes[0].Name, Is.EqualTo("attribute11"));
            Assert.That(doc.DocumentElement.Attributes[0].Value, Is.EqualTo("attribute 11 value"));
        }

        [Test]
        public void ChildNodesInnerTextAtFirstLevel()
        {
            var doc = Parse(@"<root>
              <child1>Text 1</child1>
              <child2>Text 2</child2>
            </root>");

            Assert.That(doc.DocumentElement.ChildNodes, Has.Count(2));
            Assert.That(doc.DocumentElement.ChildNodes[0].InnerText, Is.EqualTo("Text 1"));
            Assert.That(doc.DocumentElement.ChildNodes[1].InnerText, Is.EqualTo("Text 2"));
        }

        // More tests 
        .....

        private XmlDocument Parse(string xml)
        {
            var doc = new XmlDocument();

            doc.LoadXml(xml);

            return doc;
        }
    }

Such approach gives lots of advantages:

1. Easy defect location - if something wrong with attribute parsing, then only tests on attributes will fail.
2. Small tests are always easier to understand

UPD: See what Gerard Meszaros (Author of xUnit Test Patterns book) says about topic: xunitpatterns

One possibly contentious aspect of Verify One Condition per Test is what we mean by "one condition". Some test drivers insist on one assertion per test. This insistence may be based on using a Testcase Class per Fixture organization of the Test Methods and naming each test based on what the one assertion is verifying(E.g. AwaitingApprovalFlight.validApproverRequestShouldBeApproved.). Having one assertion per test makes such naming very easy but it does lead to many more test methods if we have to assert on many output fields. Of course, we can often comply with this interpretation by extracting a Custom Assertion (page X) or Verification Method (see Custom Assertion) that allows us to reduce the multiple assertion method calls into one. Sometimes that makes the test more readable but when it doesn't, I wouldn't be too dogmatic about insisting on a single assertion.

Answer 2

You might want to also use your own assertions (this is taken from your own question):

attribute11 and attribute12 belong to element1

('attribute11 ', 'attribute12').belongsTo('element1');

or

('element1 attribute11').length

The

BTW, this is similar to jQuery. You store this string in an complex graph repository. How would you unit test a very complex graph-connected database?

Answer 3

To elaborate a bit on Ian's terse answer: make that bit of XML the setup, and have separate individual tests, each with their own assertion. That way you're not duplicating the setup logic, but you still have fine-grained insight into what's wrong with your parser.

Answer 4

I had a similar kind of requirement , where i wanted to have 1 Assert for various input sets. please chekout the link below , which i blogged .

Writing better unit tests

This can be applied for your prob as well. Construct a factory class , that contains the logic for constructing 'complex' input sets . The unit test case has only one Assert.

Hope this helps.

Thanks , Vijay.

Answer 5

Use multiple tests. The same restrictions apply. You should test some normal operational cases, some failing cases, and some edge cases.

In the same way that you can presume that if sum(x, y) works for some values of x it will work with other values, you can presume that if an XML parser can parse a sequence of 2 nodes, it can also parse a sequence of 100 nodes.

Answer 6

Use Nunit Fluent Syntax

Assert.That( someString,
      Is.Not.Null
      .And.Not.Empty
      .And.EqualTo("foo")
      .And.Not.EqualTo("bar")
      .And.StartsWith("f"));

Answer 7

Multiple tests.