How would you use TDD with a neural network or a GUI application?

Question

I'm new to Test Driven Development and trying to understand the basics. For simple methods it seems straightforward, but suppose I'm working on taking some points of a .txt file and drawing them on an image. Is there a way to test that this is doing what I want without having to actually check it?

Or, suppose I have to test something that isn't deterministic like a neural network, which can sometimes get 55% accuracy or 82% in other occasions. How can I test those? I realize I can test the simpler methods, but still, there's always the chance of things breaking elsewhere.

Answer 1

There are a few strategies you can use to deal with each of the concerns you have brought up. I'll start with the most important, Unit Testing should not be used to verify facts outside of the domain of your application. That means that you would not unit test a third party GUI library, nor would you use it to verify that a mathematical theory holds true. You don't test that 5+5 actually equals 10, you only need to prove that your implementation of addition provides some correct results.

So you would not normally use unit testing to show that your neural network converges as expected. That behavior is known to work. You need only prove that your code implements the constraints and behaviors necessary for the external theory of neural networks to apply.

What this all boils down to is that make sure you are testing the right things. Make sure you are unit testing only units.

The next strategy, used especially for higher abstraction layers in your application, is to substitute layers lower than the unit under test with special versions that exhibit known behaviors. This is roughly the idea of Mock testing. For instance, when testing something that interacts with a database, you don't normally want to allow the code under test to interact with a real database. For one thing, databases retain state, and you don't want that in unit testing, tests should be atomic and independent. Another issue is that an external database can cause the test to fail for reasons outside the control of the module under test.

The solution is to inject a false database layer that responds to the same API as a real database, but provides prepared answers under the control of the unit test itself.

Extending that to the Neural net example, suppose we wish to test the back-prop algorithm. The unit test can inject a mock network that provides outputs that vary from the training data in specific ways, and the unit test verifies that the back-prop code correctly modifies the correct weights. This way we can eliminate the tests sensitivity to the behavior of network, thus isolating only the back-prop code being tested.

Specifically for testing user facing code, the usual process is to do integration testing, which employs special knowledge of a particular UI (Qt or html or whatever) to interact with the application. This mostly operates by stimulating UI events associated with a particular feature and then checking that the events propagated into the proper changes in the model layer