CODEWITHSUNDEEP
javaunit-testingparsingtestingjunit
waylonion
waylonion

Reputation: 6976

Reduce JUnit Output

I am programmatically running a JUnit test with the following method:

String res = runProcess("java -classpath " + dirPath + ";" + classpath + " org.junit.runner.JUnitCore "+ getClassName(filePath));

private static String runProcess(String command) throws Exception {
    Process process = Runtime.getRuntime().exec(command);
    String inputStr = IOUtils.toString(process.getInputStream());
    process.waitFor();
    return inputStr;
}

However, it outputs a lot of additional information such as the number of failures, time it took to run, and junit information.

A sample output looks like this:

JUnit version 4.12
.E
Time: 0.011
There was 1 failure:
1) test10(ErrorTestLang)
java.lang.AssertionError: Contract failed: compareTo-equals on fraction1 and fraction4
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at ErrorTestLang.test10(ErrorTestLang.java:32)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
    at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
    at org.junit.runner.JUnitCore.main(JUnitCore.java:36)

FAILURES!!!
Tests run: 1,  Failures: 1

My goal is to programmatically run the JUnit test suite multiple times as I programmatically modify it and ensure that the results are still the same. That is, all of the same assertions fail/pass even if I remove some lines of unused/"dead" code.

Thus, I am interested in the part of the output that is:

java.lang.AssertionError: Contract failed: compareTo-equals on fraction1 and fraction4

Since I will be comparing that with other results and don't want to consider the details such as time it took to run or line numbers.

Is there a flag to make the JUnit output more concise?

Upvotes: 1

Views: 354

Answers (2)

Mark Bramnik
Mark Bramnik

Reputation: 42541

I don't think that you can make JUnit more concise this way, after all you're running an external Java program, it produces the exception and that exception appears on screen. Technically its not even exception but a java.lang.Error

This is how org.junit.Assert#fail is implemented:

public static void fail(String message) {
    if(message == null) {
        throw new AssertionError();
    } else {
        throw new AssertionError(message);
    }
}

Of course you parse the output of the stream, but let me suggest you a different approach:

Instead of running JUnitCore directly probably you can run it through your wrapper, and in this wrapper you can add a listener that will track the test execution and will be able to react on events like failures/successful execution.

There is a concept of RunListener in Junit that can handle that:

 public class MyWrapperToRun {
   public void main(String... args) {
      JUnitCore core= new JUnitCore();
      core.addListener(new MyWrappingListener());
      core.run(MyTestClass.class);   // I don't have access to MyTestClass.class
   }
 } 

 public class MyWrappingListener implements RunListener {
   // react on events emitted by junit in methods of this listener
 }

In a listener implementation you can store the results in file or even database for future comparison.

Upvotes: 2

waylonion
waylonion

Reputation: 6976

Currently, running the JUnit test outputs as described above produces a consistent output.

We can obtain information about the method that has a failure as well as the failing assertion.

This is indicated by x) where x is xth failing assertion.

/**
 * Minimize the String output obtained from running a JUnit test suite
 * 
 * @param input
 *            The String produced from running a JUnit test suite
 * @return A String that has been minimized to contain the method that
 *         contains the failing assertion and the assertion that failed
 */
private static String minimizeJUnitOutput(String input) {
    Scanner scn = new Scanner(input);

    // JUnit output starts with index 1 for first failure
    int index = 1;
    // String to represent the result that we return
    String res = "";
    // Continue until we finish processing the input String
    while (scn.hasNextLine()) {
        String line = scn.nextLine();
        // Look for a String of the form i) where i is the current failing
        // test index
        if (line.startsWith(index + ") ")) {
            res += line + "\n" + scn.nextLine() + "\n";
            index++;
        }
    }
    scn.close();
    return res;
}

The preceding solves this problem relatively well.

Upvotes: 0

Related Questions