JUnit: Can (should) this be done?

Question

I'm trying to write some UI-Tests for a web app, but there are a couple of complications that I hope you could help me resolve.

First of all the app has two modes. One of the modes is 'training' and another is 'live'. In live mode the data is taken straight from our database and any changes made by my tests are reflected in the database, before running each test in 'live' I need to generate the test data as it's different every time (I already have that logic in place). In training mode all of the data is static so the same test data is used every time.

As the UI is the same in each mode I want to run the tests once against each mode, however due to certain peculiarities some of my tests can only be ran in live mode.

How I'd like to do this ideally is to have split my Tests into a couple of classes, say: UserTests, PaymentTests, LiveOnlyTests, etc. And have two different classes (extending a common interface) that provide the test data.

Then out of those pieces I'd like to build 2 different test suites, one for each mode. The test suites would accept the object with data that I pass them and execute the tests.

Does anyone know if this or something that achieves a similar effect is possible to do in JUnit (or any other java-test framework for that matter). The only way I can see this getting done is through writing my own test-runner, but I'd rather avoid that.

Thanks in advance.

Answer 1

What I would do is look at a combination of Parameterized and Assume.

Parameterized allows you to define a set of test data for a test. You could return the data set for the training/live (based perhaps on a System property).

Assume allows you to test something, and if the assumption is false, it aborts the test (with a failed assumption, NOT a failure).

You could also use Categories, but running them with Parameterized is harder, because have to combine two runners.

So, assuming parameterized & assume:

@RunWith(Parameterized.class)
 public class MyTest {
  @Parameters
  public static List<Object[]> data() {
    // return different data, depending upon the value of System.getProperty("testType");
    return Arrays.asList(new Object[][] {
      { 0, 0 }, { 1, 1 }, { 2, 1 }, { 3, 2 }, { 4, 3 }, { 5, 5 }, { 6, 8 }
    });
  }

  private int fInput;
  private int fExpected;

  public MyTest(int input, int expected) {
    fInput= input;
    fExpected= expected;
  }

  @Test
  public void testTraining() {
    Assume("training".equals(System.getProperty("testType"));

    // test which only runs for training, using the data from data()
  }

  @Test
  public void testBoth() {
    // test which only runs for both, using the data from data()
  }
 }

With some combination of the above, you can probably get what you want.