CODEWITHSUNDEEP
unit-testinggroovyconfigurationspock
Jan Thomä
Jan Thomä

Reputation: 13604

Dynamically excluding classes from Spock tests

We want to exclude a particular set of test classes from our Spock tests, depending on whether or not some system properties are set. Right now we got some code like:

runner {
    // skip all slow tests automatically unless test.include.slow=true
    if (!Boolean.getBoolean('test.include.slow')) { exclude Slow }

    // skip all api tests unless test.include.api=true
    if (!Boolean.getBoolean('test.include.api')) { exclude ApiTest }
}

in our SpockConfig.groovy. Problem is, that the second call to exclude actually overwrites the excludes defined in the first call. We also tried to build an array of classes and handle that over to the exclude function like this:

runner {

    Class[] toExclude = []

    // skip all slow tests automatically unless test.include.slow=true
    if (!Boolean.getBoolean('test.include.slow')) { toExclude << Slow }

    // skip all api tests unless test.include.api=true
    if (!Boolean.getBoolean('test.include.api')) { toExclude << ApiTest }

    exclude toExclude
}

This however yields very strange exceptions:

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.junit4.IdeaSuite.getDescription(IdeaSuite.java:55)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:43)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: groovy.lang.MissingMethodException: No signature of method: SpockConfig.runner() is applicable for argument types: (SpockConfig$_run_closure1) values: [SpockConfig$_run_closure1@66d33a32]
Possible solutions: run(), run(), run(java.io.File, [Ljava.lang.String;), use([Ljava.lang.Object;)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:78)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
    at SpockConfig.run(SpockConfig.groovy:4)
    at org.spockframework.builder.DelegatingScriptBlueprint.evaluate(DelegatingScriptBlueprint.java:33)
    at org.spockframework.builder.Sculpturer.$form(Sculpturer.java:32)
    at org.spockframework.builder.GestaltBuilder.build(GestaltBuilder.java:19)
    at org.spockframework.runtime.ConfigurationBuilder.build(ConfigurationBuilder.java:30)
    at org.spockframework.runtime.RunContext.<init>(RunContext.java:54)
    at org.spockframework.runtime.RunContext.createBottomContext(RunContext.java:150)
    at org.spockframework.runtime.RunContext.get(RunContext.java:130)
    at org.spockframework.runtime.Sputnik.runExtensionsIfNecessary(Sputnik.java:86)
    at org.spockframework.runtime.Sputnik.getDescription(Sputnik.java:55)
    at org.junit.runners.Suite.describeChild(Suite.java:123)
    at com.intellij.junit4.IdeaSuite.describeChild(IdeaSuite.java:68)
    at com.intellij.junit4.IdeaSuite.getChildren(IdeaSuite.java:85)
    at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:351)

So what would be to proper way to handle such exclusions in SpockConfig.groovy?

Upvotes: 2

Views: 2956

Answers (3)

Phil
Phil

Reputation: 36289

Spock supports JUnit test rules, which can provide a really clean way of doing this. For example, your feature could just use an annotation to specify when to run:

@SlowTest
def "run spec if Java system property 'test.include.slow' is true"() {
    expect:
    true
}

@ApiTest
def "run spec if Java system property 'test.include.api' is true"() {
    expect:
    true
}

First, you will need to create your annotations:

SlowTest.java:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface SlowTest {
}

ApiTest.java:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface ApiTest {
}

Next, create your test rule:

AnnotationsTestRule.java:

public class AnnotationsTestRule implements TestRule {

    private final boolean mApiTest;
    private final boolean mSlowTest;

    public AnnotationsTestRule() {
        mApiTest = Boolean.getBoolean("test.include.api")
        mSlowTest = Boolean.getBoolean("test.include.slow")
    }

    @Override
    public Statement apply(Statement base, Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                if (description.getAnnotation(ApiTest.class) != null && !mApiTest || description.getAnnotation(SlowTest.class) != null && !mSlowTest) {
                    //Skip test when the annotation is present but not the correlating system property
                    return;
                }
                base.evaluate();
            }
        }
    }

}

Finally, add the annotations mentioned in the first part of this answer on each of your features that need this. For Specifications where these are used, you also need to declare the test rule. You can do this simply by adding this line to your groovy file:

@Rule AnnotationsTestRule mAnnotationsTestRule

Upvotes: 0

WLPhoenix
WLPhoenix

Reputation: 304

Spock now has the @IgnoreIf annotation to do exactly this.

Above each test you wish to exclude, you can annotate the condition, including system properties, environment variables, and java version information.

@IgnoreIf({ !Boolean.valueOf(properties['test.include.slow']) })
def "run spec if Java system property 'test.include.slow' is true"() {
    expect:
    true
}

Upvotes: 3

Jan Thom&#228;
Jan Thom&#228;

Reputation: 13604

Ok we found some solution to this:

runner {
    // skip all slow tests automatically unless test.include.slow=true
    if (!Boolean.getBoolean('test.include.slow')) { exclude.annotations << Slow }

    // skip all api tests unless test.include.api=true
    if (!Boolean.getBoolean('test.include.api')) { exclude.annotations << ApiTest }
}

Still wondering if this is the correct way or just some crappy hack.

Upvotes: 3

Related Questions