CODEWITHSUNDEEP
junit5
mkobit
mkobit

Reputation: 47269

How to test extension implementations

There are several extension points available in the JUnit 5 API.

For example:

Nearly all of these extensions take some form of ExtensionContext which provides access to the test class, test method, test instance, ability to publish report entries, store values, and other capabilities. A lot of these operate directly on class instances

Most implementations will probably use some combination of reflection using ReflectionSupport and annotation discovery using AnnotationSupport, which are all static methods, which makes them difficult to mock.

Now, say I have written an implementation of the ExecutionCondition, or an extension that implements both the BeforeEachCallback and AfterEachCallback - how do I effectively test them?

I guess I could mock ExtensionContext, but that doesn't seem like a good way to effectively exercise the different code paths.

Internally to JUnit, there is the ExecutionEventRecorder and it is used along with the JupiterTestEngine implementation for execution (which is @API(status=INTERNAL)) to select and execute the tests, and then perform assertions on the resulting events.

I could make use of the same pattern because of the public visibility on the JupiterTestEngine.

Here are a few concrete examples to demonstrate:

  1. An ExecutionCondition that enables tests based on system properties. This can possibly be tested with some reflection and using TestInfo in a @BeforeEach or @AfterEach style, but it seems more complicated to deal with and possible issues when parallel test execution comes. This example shows an example of "how do I provide a real-life ExtensionContext and assert on the result".

    @Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@ExtendWith(SystemPropertyCondition.class)
public @interface SystemProperty {
  String key();
  String value();
}

public class SystemPropertyCondition implements ExecutionCondition {
  @Override
  public ConditionEvaluationResult evaluateExecutionCondition(final ExtensionContext context) {
    return context.getTestMethod()
        .flatMap(element -> AnnotationSupport.findAnnotation(element, SystemProperty.class))
        .map(systemProperty -> {
          if (systemProperty.value().equals(System.getProperty(systemProperty.key()))) {
            return ConditionEvaluationResult.enabled("Property is available");
          } else {
            return ConditionEvaluationResult.disabled("Property not equal");
          }
        })
        .orElseGet(() -> ConditionEvaluationResult.enabled("No annotations"));
  }
}

  2. An ExecutionCondition that only runs on the provided day of the week. How would a test clock be injected? How would you test if the condition was the expected result? I guess some state/implementations can be placed into a ExtensionContext.Store which would allow for some sort of state being passed between extensions (like with a BeforeEachCallback as well, which might be the right path, but that is going to depend on the ordering of extension execution. I don't believe BeforeEachCallback is called before an ExecutionCondition, so that path might not be the right one. This example shows both the "how do I inject dependencies" and also shows the same issue as the previous example of providing the ExtensionContext and asserting on the result.

    @ExtendWith(RunOnDayCondition.class)
public @interface RunOnDay {
  DayOfWeek[] value();
}

final class RunOnDayCondition implements ExecutionCondition {

  private static final ConditionEvaluationResult DEFAULT = disabled(
      RunOnDay.class + " is not present"
  );

  @Override
  public ConditionEvaluationResult evaluateExecutionCondition(final ExtensionContext context) {
    return context.getElement()
        .flatMap(annotatedElement -> findAnnotation(annotatedElement, RunOnDay.class))
        .map(RunOnDay::value)
        .map(RunOnDayCondition::evaluateIfRunningOnDay)
        .orElse(DEFAULT);
  }

  private static ConditionEvaluationResult evaluateIfRunningOnDay(final DayOfWeek[] days) {
    // TODO: How would you inject a test clock?
    final DayOfWeek currentDay = LocalDate.now().getDayOfWeek();
    final boolean runningInday = Stream.of(days).anyMatch(currentDay::equals);

    if (runningInday) {
      return enabled("Current day is " + currentDay + ", in the specified days of " + Arrays.toString(days));
    } else {
      return disabled("Current day is " + currentDay + ", not in the specified days of " + Arrays.toString(days));
    }
  }
}

  3. An extension that sets up a temporary directory, provides it as a parameter, and then cleans it up after the test. This would use ParameterResolver, BeforeEachCallback, AfterEachCallback, and the ExtensionContext.Store. This example shows that an extension implementation may use multiple extensions points and can make use of the store to keep track of state.

Would a custom test engine for extension tests be the right approach?

So, how do I test various extension implementations without relying on internal APIs and without "duplicating" effort into mocks?

Upvotes: 2

Views: 1030

Answers (2)

JojOatXGME
JojOatXGME

Reputation: 3296

Since JUnit 5.4, you can use Test Kit to test your extensions. The feature seems to become stable with JUnit 5.7. (Note that the API has changed between these versions.)

With Test Kit, you can execute mock-tests within actual tests and perform assertions on the result. Note that you probably need to prevent your mock tests from beeing discovered directly. Issue #1779 of JUnit 5 might help you with that.

Upvotes: 1

Carlo Bellettini
Carlo Bellettini

Reputation: 1180

I know that is not a complete answer (but I am on mobile now and your question is very complex)... Try however if this project could help you.

If you limit a little your question I could try to help you better

Upvotes: 1

Related Questions