CODEWITHSUNDEEP
javascriptunit-testingcontinuous-integrationyuiassertions
op1ekun
op1ekun

Reputation: 1928

YUI.Test tests stops after ONLY ONE failed assertion

I coming from QUnit background (I've been very happy using it for many years). In my current project YUI is a framework of choice so it was reasonable to use YUI.Test for Unit Testing.

I'm using version 3.4.0.

Unfortunately I run into some unexpected "feature". According to documentation http://yuilibrary.com/yui/docs/test/#assertions:

"Note: Even though this example shows multiple assertions failing, a test will stop as soon as one assertion fails, causing all others to be skipped."

I'm used to grouping assertions under meaningful test names. If I have 5 assertions and 2 of them fail I want to see both in the test results. If there are numerous failing assertion in a single test I can't fix all of them at once. I have to "fix" the first one restest, find about another one... "fix" and retest... and so on.

It's not a big deal when I'm on my local environment. The problem starts to get serious in Continuous Integration environment. I don't want to run builds (on Bamboo/Jenkins) for every failed assertion. If I have multiple tests in a TestCase (which is highly probable...) it's a nightmare and my Team will kill after only one day ...

Currently I'm using a "workaround"... Every test has ONLY ONE assertion :( This means a lots of tests...

Is there any other way to fix this issue?

ANY HELP APPRECIATED! THANKS!

Upvotes: 1

Views: 162

Answers (1)

Clarence Leung
Clarence Leung

Reputation: 2566

What you want to do (and what we do in our own tests in YUI) is to group your different tests under a single Y.Test.Case. Here's an example, taken from the tests in Y.ArraySort:

var ArrayAssert = Y.ArrayAssert,

    suite = new Y.Test.Suite('ArraySort');

    suite.add(new Y.Test.Case({
        name: 'compare()',

        'should compare numbers': function () {
            var array = [2,1,3,5,4];
            array.sort(Y.ArraySort.compare);
            ArrayAssert.itemsAreSame([1,2,3,4,5], array, "Expected sorted numbers.");
        },

        'should compare strings': function () {
            var array = ["caa", "baa", "bba", "aba", "cba", "aaa", "abc"];
            array.sort(Y.ArraySort.compare);
            ArrayAssert.itemsAreSame(["aaa","aba","abc","baa","bba","caa","cba"], array, "Expected sorted strings.");
        },

        'should compare mixed alpha and numeric strings': function() {
            var array = ["attic", "Aardvark", "1", "0", "Zoo", "zebra"];
            array.sort(Y.ArraySort.compare);
            ArrayAssert.itemsAreSame(["0", "1", "Aardvark","attic","zebra","Zoo"], array, "Expected sorted mixed strings.");
        }
    }));

Each test inside of a Y.Test.Case is supposed to fail if one assertion fails, and you should group related tests under a single Y.Test.Case name instead. The name that a single test has should be a simple, human-readable description that describes the particular feature you're testing, and should only assert for that feature alone.

Upvotes: 1

Related Questions