CODEWITHSUNDEEP
javaunit-testingmavenjvm
Cerber
Cerber

Reputation: 2939

Test against multiple JVM or library versions during Maven build?

Context

Basically, I have a library that is used in several applications running on java 6, java 7 and now java 8.

The library also has a dependency on a creepy (yet very useful) third party library (which I won't name) - a library that doesn't care a lot about forward-compatibility. The aforementioned applications also make heavy use of this third party library but are not always able to update to the latest version. I'm used to it, so I have a lot of "if" in my code dealing with silly things.

Problem

Today I stumbled upon an annoying problem: some method in the third party library has a different behavior when running in java 7 and earlier than when running in java 8 and later.

My unit tests were designed to cover this part of my code and would have spotted the bug if they were run with the correct java / library combination.

Question 1

How can I integrate nicely in the maven build lifecycle and run my tests sequentially with java 6, java 7 and java 8? (which implies failing if any test fails with any jvm)

Question 2

Same as above but adding another sequential run with each "supported" version of the third party library ?

(which means, to be clear, that, if I "support" version 6.11, 6.12, 6.13 and 6.14 of this lib with all 3 java versions, I will run 4*3=12 times my unit tests)

Upvotes: 2

Views: 395

Answers (1)

piotrek
piotrek

Reputation: 14550

i have similar problem. i created a library that should be tested against different versions (all available in maven central) of other library and (but that's less important) jvm.

my conclusions: doing this only in maven will be veeeery cumbersome and hard, if not impossible. you will also have to encode environment/system dependencies into your build tool

as suggested, most integration servers support matrix tests. you can use it also to manually provide every single version (if you know them upfront) of library you need to test against. the library part should be doable much easier in gradle (but still no out-of-the-box support).

regarding number of tests: yes will have to run x * y times. if your code is big think about isolating the part that uses other library and run only this small part x times for each jvm - it will be a bit faster. also you can run those tests in parallel

ps: i'm thinking about writing a gradle plugin or at least a proof-of-concept build. but i don't think it will be soon. but i'm open for other contributors :)

Upvotes: 1

Related Questions