Parameterized test case classes in JUnit 3.x

Question

I have a JUnit 3.x TestCase which I would like to be able to parameterize. I'd like to parametrize the entire TestCase (including the fixture). However, the TestSuite.addTestSuite() method does not allow be to pass a TestCase object, just a class:

   TestSuite suite = new TestSuite("suite");
   suite.addTestSuite(MyTestCase.class);

I would like to be able to pass a parameter (a string) to the MyTestCase instance which is created when the test runs. As it is now, I have to have a separate class for each parameter value.

I tried passing it an anynomous subclass:

   MyTestCase testCase = new MyTestCase() {
       String getOption() {
           return "some value";
       }
   }

   suite.addTestSuite(testCase.getClass());

However, this fails with the assertion:

   ... MyTestSuite$1 has no public constructor TestCase(String name) or TestCase()`

Any ideas? Am I attacking the problem the wrong way?

Answer 1

For Android projects, we wrote a library called Burst for test parameterization. For example

public class ParameterizedTest extends TestCase {
  enum Drink { COKE, PEPSI, RC_COLA }

  private final Drink drink;

  // Nullary constructor required by Android test framework
  public ConstructorTest() {
    this(null);
  }

  public ConstructorTest(Drink drink) {
    this.drink = drink;
  }

  public void testSomething() {
    assertNotNull(drink);
  }
}

Not really an answer to your question since you're not using Android, but a lot of projects which still use JUnit 3 do so because Android's test framework requires it, so I hope some other readers will find this helpful.

Answer 2

a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).

To solve this you just need to overwrite getName() and change the constructor in your test case class:

 private String displayName;

 public ParameterizedTest(final int value) {
     this.value = value;
     this.displayName = Integer.toString(value);
 }

 @Override
 public String getName() {
     return super.getName() + "[" + displayName + "]";
 }

Answer 3

Ok, here is a quick mock-up of how JUnit 4 runs parameterized tests, but done in JUnit 3.8.2.

Basically I'm subclassing and badly hijacking the TestSuite class to populate the list of tests according to the cross-product of testMethods and parameters.

Unfortunately I've had to copy a couple of helper methods from TestSuite itself, and a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).

Nevertheless, this seems to run fine in the text and AWT TestRunners that ship with JUnit as well as in Eclipse.

Here is the ParameterizedTestSuite, and further down a (silly) example of a parameterized test using it.

(final note : I've written this with Java 5 in mind, it should be trivial to adapt to 1.4 if needed)

ParameterizedTestSuite.java:

package junit.parameterized;

import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;

public class ParameterizedTestSuite extends TestSuite {

    public ParameterizedTestSuite(
            final Class<? extends TestCase> testCaseClass,
            final Collection<Object[]> parameters) {

        setName(testCaseClass.getName());

        final Constructor<?>[] constructors = testCaseClass.getConstructors();
        if (constructors.length != 1) {
            addTest(warning(testCaseClass.getName()
                    + " must have a single public constructor."));
            return;
        }

        final Collection<String> names = getTestMethods(testCaseClass);

        final Constructor<?> constructor = constructors[0];
        final Collection<TestCase> testCaseInstances = new ArrayList<TestCase>();
        try {
            for (final Object[] objects : parameters) {
                for (final String name : names) {
                    TestCase testCase = (TestCase) constructor.newInstance(objects);
                    testCase.setName(name);
                    testCaseInstances.add(testCase);
                }
            }
        } catch (IllegalArgumentException e) {
            addConstructionException(e);
            return;
        } catch (InstantiationException e) {
            addConstructionException(e);
            return;
        } catch (IllegalAccessException e) {
            addConstructionException(e);
            return;
        } catch (InvocationTargetException e) {
            addConstructionException(e);
            return;
        }


        for (final TestCase testCase : testCaseInstances) {
            addTest(testCase);
        }       
    }
    private Collection<String> getTestMethods(
            final Class<? extends TestCase> testCaseClass) {
        Class<?> superClass= testCaseClass;
        final Collection<String> names= new ArrayList<String>();
        while (Test.class.isAssignableFrom(superClass)) {
            Method[] methods= superClass.getDeclaredMethods();
            for (int i= 0; i < methods.length; i++) {
                addTestMethod(methods[i], names, testCaseClass);
            }
            superClass = superClass.getSuperclass();
        }
        return names;
    }
    private void addTestMethod(Method m, Collection<String> names, Class<?> theClass) {
        String name= m.getName();
        if (names.contains(name))
            return;
        if (! isPublicTestMethod(m)) {
            if (isTestMethod(m))
                addTest(warning("Test method isn't public: "+m.getName()));
            return;
        }
        names.add(name);
    }

    private boolean isPublicTestMethod(Method m) {
        return isTestMethod(m) && Modifier.isPublic(m.getModifiers());
     }

    private boolean isTestMethod(Method m) {
        String name= m.getName();
        Class<?>[] parameters= m.getParameterTypes();
        Class<?> returnType= m.getReturnType();
        return parameters.length == 0 && name.startsWith("test") && returnType.equals(Void.TYPE);
     }

    private void addConstructionException(Exception e) {
        addTest(warning("Instantiation of a testCase failed "
                + e.getClass().getName() + " " + e.getMessage()));
    }

}

ParameterizedTest.java:

package junit.parameterized;
import java.util.Arrays;
import java.util.Collection;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.parameterized.ParameterizedTestSuite;


public class ParameterizedTest extends TestCase {

    private final int value;
    private int evilState;

    public static Collection<Object[]> parameters() {
        return Arrays.asList(
                new Object[] { 1 },
                new Object[] { 2 },
                new Object[] { -2 }
                );
    }

    public ParameterizedTest(final int value) {
        this.value = value;
    }

    public void testMathPow() {
        final int square = value * value;
        final int powSquare = (int) Math.pow(value, 2) + evilState;
        assertEquals(square, powSquare);
        evilState++;
    }

    public void testIntDiv() {
        final int div = value / value;
        assertEquals(1, div);
    }

    public static Test suite() {
        return new ParameterizedTestSuite(ParameterizedTest.class, parameters());
    }
}

Note: the evilState variable is just here to show that all test instances are different as they should be, and that there is no shared state between them.

Answer 4

Rather than create a parameterized test case for the multiple/different backends you want to test against, I would look into making my test cases abstract. Each new implementation of your API would need to supply an implementing TestCase class.

If you currently have a test method that looks something like

public void testSomething() {
   API myAPI = new BlahAPI();
   assertNotNull(myAPI.something());
}

just add an abstract method to the TestCase that returns the specific API object to use.

public abstract class AbstractTestCase extends TestCase {
    public abstract API getAPIToTest();

    public void testSomething() {
       API myAPI = getAPIToTest();
       assertNotNull(myAPI.something());
    }

    public void testSomethingElse() {
       API myAPI = getAPIToTest();
       assertNotNull(myAPI.somethingElse());
    }
}

Then the TestCase for the new implementation you want to test only has to implement your AbstractTestCase and supply the concrete implementation of the API class:

public class ImplementationXTestCase extends AbstractTestCase{

    public API getAPIToTest() {
        return new ImplementationX();
    }
}

Then all of the test methods that test the API in the abstract class are run automatically.

Answer 5

If this is Java 5 or higher, you might want to consider switching to JUnit 4, which has support for parameterized test cases built in.