CODEWITHSUNDEEP
phpunit-testing
mellowsoon
mellowsoon

Reputation: 23261

Unit Testing, Coupling Between Tests, and Cascading Failures

I'm curious how people deal with coupling between tests, and "cascaing failures". I don't know what else to call it, so I'll explain with some code.

I have a simple caching class that I want to test.

class Cache
{
    private $data = array();
    public function set($key, $data)
    {
        $this->data[$key] = $data;
        return true;
    }
    public function get($key)
    {
        if (!isset($this->data[$key])) {
            return false;
        }
        return $this->data[$key];
    }
}

Now I want to test the class, and both of the methods in the class. Here's the problem; the get() method depends on the set() method. So I can write my test case in one of two ways.

class TestCache extends Test
{
    private $cache = null;
    public function setup()
    {
        $this->cache = new Cache();
    }
    public function testSet()
    {
        $result = $this->cache->set('foo', 'bar');
        $this->asserTrue($result);
    }
    public function testGet()
    {
        $result = $this->cache->get('foo');
        $this->assertEquals($result, 'bar');
    }
}

The problem here is the second test, testGet(), replies on the first test, testSet(). If testSet() fails, so will testGet(). There's a kind of coupling between the two tests. Additionally the testGet() test will fail if I change the order of the test methods. So instead I write my test case like this:

class TestCache extends Test
{
    private $cache = null;
    public function setup()
    {
        $this->cache = new Cache();
    }
    public function testSet()
    {
        $result = $this->cache->set('foo', 'bar');
        $this->asserTrue($result);
    }
    public function testGet()
    {
        $this->cache->set('foo', 'bar');
        $result = $this->cache->get('foo');
        $this->assertEquals($result, 'bar');
    }
}

Now there's no coupling between the two tests, but there's still a problem. If the Cache::set() method is failing, then the second test will also fail along with the first, because the second test's call to Cache::get() replies on Cache::set() not failing.

So I'm wondering how you typically deal with a situation like this. Do you simply accept dozens of failures in your test output because of one bad test? Do you abort further testing on the class if an important test fails?

Upvotes: 0

Views: 337

Answers (2)

Yaniro
Yaniro

Reputation: 1587

How are you running your tests? do you call TestCache::testSet() followed by TestCache::testGet()? what do assertTrue() and assertEquals() do? do they use exceptions in order to break the current execution path? if not, that would be a good way to group interdependent tests for example if Test::assertTrue() is implemented like so:

public function assertTrue( $value, $method = "" )
{
    if ( $value !== true )
    {
        throw new Exception( "assertTrue() failed on $method, Value $value does not evaluate to true" );
    }
}

Then, you couple the tests like so (inside testSet() & testGet() pass the method name "Cache::set" or "Cache::get" to the assert functions:

try
{
    TestCache::testSet();
    TestCache::testGet();
}
catch ( $e )
{
    echo( "Failed - " . $e->getMessage() );
}

Now, if testSet() fails, execution skips testGet()

Upvotes: 0

Mark Baker
Mark Baker

Reputation: 212412

This is what the @depends directive is for in phpunit: see the section on "Test Dependencies". If a dependency has failed, then subsequent tests dependent on that intial test will be skipped.

Upvotes: 2

Related Questions