CODEWITHSUNDEEP
unit-testingtdd
tanapoln
tanapoln

Reputation: 529

London TDD: dealing with inconsistency behavior when requirement has changed

I just known there are two TDD style that's is London and Classic after that my problem about TDD is more clarify so that's why i ask this question.

from this post http://codemanship.co.uk/parlezuml/blog/?postid=987
In second example, Sign-up example, if my customer, at a first time of the meeting, define that email do not contains any period eg. [email protected] is not valid because it has period between my and name, then we implement belongs to that requirement, as same as Sign-up example, and the implementation works fine for a long time.

Some day, the customer came to me and tell me that he want to allow period to presence in email, so I realize that I need to navigate to the test of Sign-up Controller and change the add more test to make sure new requirement has been met by mocking the Validator.validateEmail(email) to return false but I forgot to change Validator.validateEmail to check period then run test, all tests green (because we mock the result) but the program still bug.

I admit that the example is a small program and bug can be found quickly but what if this problem occurred somewhere that hard to find out?

I don't know if this is a dumb question, it's happen to me many times when I need to deal with complex software, when requirement has been changed I always make sure that I change all tests related but there are tests somewhere that I don't realize like the example.

So, do you have same problem? how do you deal with it?

PS. I create an code example in php with PHPUnit, see http://pastebin.com/RXP8i6Y3. Please save with AllTest.php and run with phpunit AllTest.php to see result and try to uncomment TODO code.

Update
I'm in situation that I can't trust my code when I change somethings. When application grow, number of testcase grow (sometimes more than 5000+ test), and we using mocking somewhere in test. when we're going to change something, sometimes mock object don't reflect any error in test and you can see when number of test is so much, it's make harder to maintain.

Upvotes: 0

Views: 102

Answers (1)

Liviu Mandras
Liviu Mandras

Reputation: 6627

When requirements change, you do not modify existing tests. You write a new test(s) for the new requirements and after that you implement the feature/change following the TDD cycle red-green-refactor.

When you run the entire suite of tests some of them should fail as the old logic has changed. When this happens you inspect the tests and see if it makes sense to modify them or better yet delete them as they are obsolete from the requirements perspective.

It is not very clear from your question what exactly do you want to test. But if you mock the EmailValidator than you are actually testing the controller and how that controller behaves when the result from the Validator is false and not the Validator itself.

When using mocking frameworks the actual value of parameters really counts so you need to be careful about that.

If you do TDD properly when you create a test that does not fail when first run then either you are retesting an existing requirement or your test has a bug. You must always see the test failing first when you add a new requirement otherwise you should consider that a red flag to look more carefully at the test/code or requirements.

UPDATE after link in comment

You are not testing the right things. If you want to check that the emails get validated properly you have the tests for the Validator, which are fine:

public function test_validate_with_period_return_false() {
    $inst = new Validator();
    $this->assertSame(false, $inst->validate('[email protected]'));
  }

  public function test_validate_without_period_return_true() {
    $inst = new Validator();
    $this->assertSame(true, $inst->validate('[email protected]'));
  }

  // //This is a test that support new requirement of validator
  // //even i add this test and remove test_validate_with_period_return_false
  // //how about mock of Validator in test above? how do I known where is using this class
  // //Mock don't reflect error after this method changed
  // public function test_validate_after_requirement_changed_period_must_allow() {
  //   $inst = new Validator();
  //   $this->assertSame(true, $inst->validate('[email protected]'));
  // }

After you uncomment the last test you should also replace the actual logic from the Validator and then delete the first test (that returns false when you have period in email).

Now if you want to test the controller your test should not sound like:

public function test_openUserAccount_not_allow_period_in_email()
public function test_openUserAccount_return_true_with_period_in_email()

They should sound like

public function test_openUserAccount_ThrowsException_WhenEmailIsInvalid()
public function test_openUserAccount_return_true_when_emailIsValid()

And then it does not matter what email you pass as parameter because you always return either true or false from the mock. The actual email validation is tested separately in the validator tests, now you test the controller logic when the email validator (does not matter the value of the email) returns true or false.

If you want to keep your tests as they are you should not mock the email validator. Just use the real thing. It runs completely in memory and is still fast. You give up on perfect isolation and test two classes together (the controller and the validator) but you gain on other parts, like the possibility to refactor more easily without having too many tests break.

Upvotes: 1

Related Questions