Test custom constraint annotations with JUnit 5

Question

Implementing a custom constraint annotation, like @MySize requires me testing it with unit tests to see if it functions correctly:

public class MySizeTest {
   @Test
   public void noMinMax() {
       Dummy dummy = new Dummy();
       // some asserts or so
       dummy.setMyField("");
       dummy.setMyField(null);
       dummy.setMyField("My text");
   }

   @Test
   public void onlyMin() {
       // change @MySize to have min: @MySize(min = 1)
       ... how?
       ... then test with some setMyField:
       Dummy dummy = new Dummy();
       // some asserts or so
       dummy.setMyField("");
       dummy.setMyField(null);
       dummy.setMyField("My text");
   }

   @Test
   public void onlyMax() {
       // change @MySize to have max: @MySize(max = 50)
       ...
   }

   @Test
   public void bothMinMax() {
       // change @MySize to have min and max: @MySize(min = 1, max = 50)
       ...
   }

   private class Dummy {
       @MySize()
       String myField;

       public String getMyField() {
           return myField;
       }

       public void setMyField(String myField) {
           this.myField = myField;
       }
    }
}

I assume this has to be done with reflection, but I have no idea how.

Answer 1

Basicly don't have to use reflection just create a Validator instance and use that for validating.

For examaple:

When the annotation is:

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = MyValidator.class)
public @interface MyAnnotation {
    String message() default "Invalid value (it must be foo)";
    Class<?>[] groups() default {};
    Class<? extends Payload>[] payload() default {};

}

and the related validator is:

public class MyValidator implements ConstraintValidator<MyAnnotation, String> {
    @Override
    public boolean isValid(String s, ConstraintValidatorContext constraintValidatorContext) {
        if (null == s) return true;
        return "foo".equalsIgnoreCase(s);
    }
}

Then the tests sould be like these:

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class MyValidatorTest {

    private Validator validator;

    @BeforeAll
    void init() {
        ValidatorFactory factory = Validation.buildDefaultValidatorFactory();
        validator = factory.getValidator();
    }

    private static class TestObject {

        @MyAnnotation
        private String testField;

        TestObject() {
            this(null);
        }

        TestObject(String value) {
            testField = value;
        }

        public String getTestField() {
            return testField;
        }

        public void setTestField(String testField) {
            this.testField = testField;
        }
    }

    @Test
    void shouldValidForNullValue() {
        var obj = new TestObject();
        var violations = validator.validate(obj); // Set<ConstraintViolation<TestObject>>
        Assertions.assertTrue(violations.isEmpty(), String.format("Object should valid, but has %d violations", violations.size()));
    }

    @Test
    void shouldValidForFooValue() {
        var obj = new TestObject("foo");
        var violations = validator.validate(obj); // Set<ConstraintViolation<TestObject>>
        Assertions.assertTrue(violations.isEmpty(), String.format("Object should valid, but has %d violations", violations.size()));
    }

    @Test
    void shouldInvalidForBarValue() {
        var obj = new TestObject("bar");
        var violations = validator.validate(obj); // Set<ConstraintViolation<TestObject>>
        Assertions.assertEquals(1, violations.size());
    }
}

Update (2020.05.21.) - Using attributes and AnnotationFactory

Based on comments I've updated my answer. If you want to test only the validation logic then just create an Annotation instance and call the isValid method which is returns true or false

Hibernate Validator provides AnnotationFactory.create(...) method to make annotaion instance. After that you can create an instance of your custom validator and call initialize and isValid methods in your test case.

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Constraint(validatedBy = MyHasAttributesValidator.class)
public @interface MyAnnotationHasAttributes {
    String message() default "Invalid value (it must be foo)";
    Class<?>[] groups() default {};
    Class<? extends Payload>[] payload() default {};

    int attributeOne() default 10;
    int attributeTwo() default 20;

}

related validator:

public class MyHasAttributesValidator implements ConstraintValidator<MyAnnotationHasAttributes, String> {
    private MyAnnotationHasAttributes ann;

    @Override
    public void initialize(MyAnnotationHasAttributes constraintAnnotation) {
        ann = constraintAnnotation;
    }

    @Override
    public boolean isValid(String s, ConstraintValidatorContext constraintValidatorContext) {
        if (null == s) return true;
        return s.length() >= ann.attributeOne() && s.length() < ann.attributeTwo();
    }
}

and the modified test (which has failing assertion):

public class HasAttributeValidatorTest {

    private MyAnnotationHasAttributes createAnnotation(Integer one, Integer two) {
        final Map<String, Object> attrs = new HashMap<>();
        if (null != one) {
            attrs.put("attributeOne", one);
        }
        if (null != two) {
            attrs.put("attributeOne", two);
        }
        var desc = new AnnotationDescriptor.Builder<>(MyAnnotationHasAttributes.class, attrs).build();
        return AnnotationFactory.create(desc);
    }

    @ParameterizedTest
    @MethodSource("provideValues")
    void testValidator(Integer one, Integer two, String input, boolean expected) {
        MyAnnotationHasAttributes ann = createAnnotation(one, two);
        MyHasAttributesValidator validator = new MyHasAttributesValidator();
        validator.initialize(ann);
        var result = validator.isValid(input, null);
        Assertions.assertEquals(expected, result, String.format("Validation must be %s but found: %s with params: %d, %d, %s", expected, result, one, two, input));
    }

    private static Stream<Arguments> provideValues() {
        return Stream.of(
                Arguments.of(null, null, null, true),
                Arguments.of(null, 20, "foo", true),
                Arguments.of(null, null, RandomStringUtils.randomAlphabetic(30), false)
        );
    }
}

Limitations of this solution

Vendor lock

In this case your test using Hibernate Validator which is a specific implementation if the Bean Validation standards. Honestly I don't think it's a huge problem, because Hibernate Validator is the refecerence implementation and the most popular bean validation library. But technically it's a vendor lock.

Cross field validation is unavailable

This soulution works only in one-field situations. If you have e.g a cross-field validator (e.g. password and confirmPassword matching) this example won't fit.

Type independent validation needs more work

Like previously mentioned @Size annotation belongs to several different validator implementations based on type (primitives, collections, string, etc.). Using this solution you always have to chose the certain validator manually and test it.

Only the isValid method can be tested

In this case you won't be able to test another things just the isValid method. I mean e.g. error message has expected format and parameters or something like this.

In sort, I know creating many different fields with different annotation attributes is boring but I strongly prefer that way because you can test everything you need about your validator.