CODEWITHSUNDEEP
unit-testing
bernie2436
bernie2436

Reputation: 23921

How do I write "dirty" unit tests?

I am reading Code Complete. In that book, Steve McConnell warns that "Developer tests tend to be 'clean tests.' Developers tend to test for whether the code works (clean test) rather than test for all the ways the code breaks (dirty tests)."

How do I write a test for the way the code breaks? I mean, I can write tests for bad input and make sure that it is blocked correctly. But aside from that, what sorts of things should I be thinking about? What does McConnell mean here? I am comfortable with basic unit testing but trying to master it.

Upvotes: 4

Views: 1219

Answers (3)

Péter Török
Péter Török

Reputation: 116286

I think you are on the right path here. Tests to prove that the code works would call a method with sensible, meaningful and expected inputs, with the program in normal state. While tests to break the code try to think "out of the box" regarding that piece of code, thus use any sort of senseless or unexpected input.

What is IMHO important though is to understand that the two thought processes are very different. When a developer writes code in TDD fashion, (s)he tends to focus on the various bits of functionality to implement in the code, and the tests to prove that this and that bit of functionality or use case works as specified. Tests created this way are what McConnell calls "clean tests".

Thinking about how a piece of code could be broken requires a very different thought process and different experience too. It requires looking at your methods and APIs from a different angle, e.g. temporarily putting aside what you know about the aim of these methods and parameters, and focusing only what is technically possible to do with them. Also to think about all the - often implied - preconditions or dependencies required for this method to work correctly. Does it depend on a configuration parameter read from DB? Does it write to the file system? Does it call another component, expecting it to having been initialized properly beforehand? Does it use large amounts of memory? Does it display a message on a GUI?... And what if one or more of these doesn't hold?

All this leads to important questions: how should your method handle such dirty cases? Should it crash? Throw an exception? Continue as best as it can? Return an error code? Log an error report?... All these little or bigger decisions are actually quite important for defining the contract of a method or an API correctly and consistently.

Kent Beck talks about switching between wearing "developer hat" and "tester hat" in the same sense. Fluently switching viewpoints and thought processes this way requires practice and experience.

Upvotes: 6

JeffH
JeffH

Reputation: 10482

Elisabeth Hendrickson, of Test Obsessed, has a Test Heuristics Cheat Sheet in which she lists all sorts of testing approaches. The document is broadly about testing, but the section called "Data Type Attacks" has lots of specific examples that "unhappy path" unit tests could look at.

For example, here are her ideas about ways to test paths and files:

Long Name (>255 chars)
Special Characters in Name (space * ? / \ | < > , . ( ) [ ] { } ; : ‘ “ ! @ # $ % ^ &)
Non-Existent
Already Exists
No Space
Minimal Space
Write-Protected
Unavailable
Locked
On Remote Machine
Corrupted

Upvotes: 1

k.m
k.m

Reputation: 31464

What author most likely meant by clean test is test that verifies only happy path of method execution.

Testing happy path is usually easiest and people might tend to think that since it works, their job with writing tests is done. This rarely is the case. Consider:

public void SaveLog(string entry)
{
    var outputFile = this.outputFileProvider.GetLogFile();
    var isValid = this.logValidator.IsValid(outputFile);
    if (isValid)
    {
        this.logWriter.Write(outputFile, entry);
    }
}

Happy path testing would be simply assuming all the dependencies (outputFileProvider, logValidator, logWriter) worked and write is indeed taking place. However, there's high chance something might break along the way, and those paths should be tested too. Like:

  • outputFileProvider fails to acquire output file
  • outputFile turns out to be invalid
  • logValidator fails with its own exception
  • logWriter fails to write

Just to name a few! There's lot more to unit testing than simply checking happy path, but unfortunately this is often the case.

Upvotes: 1

Related Questions