.NET UI Testing (both Unit and Integration)

Question

I'm tasked with coming up with a solution to effectively unit test and integration test at the UI level. Unfortunately, we have a lot of code-behind in windows forms. We also have a brownfield project to where we are slowly moving to WPF (new features are in WPF and moving old features to WPF when they warrant major changes).

Everything in WPF is unit tested all the way down (except the database). I'm using a MVVM approach, so almost all of the code is there.

However, testing before deployment takes a lot of time and we need a way to automate most of it.

That said, this testing that I'm referring to needs to happen at the UI.

The windows forms parts will have to be integration tested because part of the logic is done in the code-behind and part in the database.

The WPF windows should be unit-testable at the UI level, but also integration tested.

I know that is a lot to swallow for one question, but does anyone have any recommendations?

Answer 1

I totally agree with Torbjorn's answer but wanted to add a few points:

Start small

The Page Object pattern is a great way to simplify your tests, but you will find that it takes a long time to get the abstractions right. Start by abstracting what you need and slowly add to it over time.

Add value

Don't go overboard and try to write end-to-end regression tests. Instead, focus on writing tests that add value. For example, a single test that demonstrates that the application launches without errors is immensely useful and can provide early feedback on your build process. Stem out from there.

Balance "Deep" versus "Shallow"

There are a few different philosophies for testing the user interface. Establish a mix between them.

The obvious approach is to test the application with production-like settings to demonstrate that the application works "front-to-end". These are "deep" integration tests that exercise all parts of the code and are useful. They can also be painfully slow because they typically depend on external services, etc. Often, to be reliable the application must restarted between test-runs to ensure a valid environment.

A slight modification of that approach is to test the application with stubbed-out services (a fake product catalog, a fake authentication provider, etc). These are "shallow" tests that demonstrate that the user-interface works when integrated together. They typically run a bit faster because they won't have the same physical constraints such as network latency to consider. You can focus more on presentation specifics and other edge cases.

A further modification is to isolate parts of the user-interface and run them in a test harness. These tests will run much faster than the previous approaches because they won't have the same overhead of starting the entire application. Use these tests to assert colors and highly specialized presentation concerns.

Iterate when stable

If you intend on writing functional tests to replace manual regression testing you may find it best to wait until development has stabilized that feature before writing automation for it. If you start to write automation during development, you'll be constantly re-writing tests. If you want to automate during development, remember: start small.

Get early feedback

Automated testing of the UI (also known as Functional Testing) is useful, but it can be very, very slow. (I've seen runs taking several hours to complete.) If you run the entire test suite once a day, you're going to find that the feedback loop is too long which leads to false-positives, maintenance issues, etc.

Try to integrate the functional testing into the build process if possible. If the test suite takes too long, find a way to integrate some of the tests so that your build pipeline can validate important tests as part of the build.

Answer 2

(This answer is about how to integration test your code: WinForms and WPF.)

I don't have a patented solution for you, only the advice to put a lot of time and energy into making the tests easy to maintain.

Make sure you can modify and run the tests from within Visual Studio.

The extra burdon of having to start a different tool for the tests will probably be enough for developers not to do it (I've seen it happen). So try to find a UI testing tool that allows this.

Don't record the tests.

While comfortable, the resulting code of recorded tests is hard to read and understand and will most likely contain a lot of insignificant details. If you need to make changes to the tests you will probably have to re-record them. The tests will be more of a documentation of how the system works rather than a specification of how you want it to work.

Create shared infrastructure for your tests.

Create abstractions for the windows/forms in objects that represent the windows' behaviours. This way you can hide implementation details in these objects, making the actual tests compact and easy to read. Once you have this infrastructure, it's easy to add new tests. Also, if implementation details change you only have one place to make changes to the tests: in the infrastructure.

In the web tests world, this is called page object pattern.

Example of a test in C#:

ApplicationFake application = new ApplicationFake();

// ApplicationFake.Start() starts the application resulting in that main form opens.
// Start() returns an object that represents the actions that the user can perform on
// the main form.
// The Start() method might contain an assert that the main form was actually opened.
MainFormFake mainForm = application.Start();

// Performing an action that opens a new form returns a representation of the new form,
// in this case the object SomeFormFake.
SomeFormFake someForm = mainForm.ClickOpenSomeFormButton();

// The form fakes exposes properties that returns the forms' observable state.
Assert.That(someForm.Title, Is.EqualTo("Some Form's Title"));