Why is Maven not resolving all dependencies for commons-configuration?

Question

Summary

When trying XMLConfiguration configuration = new XMLConfiguration("config/config.xml"); with only commons-configuration 1.10 I need to add more depencies (namely commons-collections not newer than 3.2.1) to my maven setup. Why is that so and why doesn't maven simply resolve all needed dependencies?

Details

I am trying to get commons-configuration to work. First I wanted to use the latest version, 2.0-alpha2, which didn't work well at all since I was unable to configure Maven to download the correct ressources - but that is another story.

After I found out that version 1.10 is in fact "one point ten" (not "one point one zero") and thus the latest version of commons-configuration 1 (and covered by the tutorials), I decided to give it a try instead.

For my maven dependencies (integrated in eclipse) I used:

<dependency>
    <groupId>commons-configuration</groupId>
    <artifactId>commons-configuration</artifactId>
    <version>1.10</version>
</dependency>

However, when trying out this example:

package main;

import java.util.Iterator;

import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.XMLConfiguration;

public class ConfigurationTest {
    public static void main(String... args) {
        try {
            XMLConfiguration configuration = 
                    new XMLConfiguration("config/config.xml");
            Iterator<String> iterator = configuration.getKeys();
            while (iterator.hasNext()) {
                System.out.println(iterator.next());
            }
        } catch (ConfigurationException e) {
            e.printStackTrace();
        }
    }
}

with the following config.xml:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<configuration>
  <property>value</property>
  <nestedproperty>
    <arrayvalue>0,1,2,3,4</arrayvalue>
    <property>anothervalue</property>
  </nestedproperty>
</configuration>

I got the error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/CollectionUtils
    at org.apache.commons.configuration.XMLConfiguration.constructHierarchy(XMLConfiguration.java:640)
    at org.apache.commons.configuration.XMLConfiguration.initProperties(XMLConfiguration.java:596)
    at org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:1009)
    at org.apache.commons.configuration.XMLConfiguration.load(XMLConfiguration.java:972)
    at org.apache.commons.configuration.XMLConfiguration$XMLFileConfigurationDelegate.load(XMLConfiguration.java:1647)
    at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:324)
    at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:261)
    at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
    at org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:184)
    at org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.<init>(AbstractHierarchicalFileConfiguration.java:95)
    at org.apache.commons.configuration.XMLConfiguration.<init>(XMLConfiguration.java:261)
    at main.ConfigurationTest.main(ConfigurationTest.java:12)

I first hoped they (not me, of course) just screwed up some maven dependencies and since I wouldn't bother which version to use anyway anymore (I didn't get 2.0 to work, remember?) I decided to go down to version 1.9 by replacing the maven dependency with:

<dependency>
    <groupId>commons-configuration</groupId>
    <artifactId>commons-configuration</artifactId>
    <version>1.9</version>
</dependency>

That solved the problem pretty well, the test case is running:

property
nestedproperty.arrayvalue
nestedproperty.property

But when I tried to implement a similar example to the one referenced in Very simple Apache-commons configuration example throws NoClassDefFoundError and its follow-up question I got the exact same error which is referenced there - but the solution, importing org.apache.commons.beanutils.PropertyUtils is not working as I am missing the beanutils. So basically by downgrading I just switched from the error of missing the collections to missing beanutils.

There is a dependency overview where you can see which dependencies are used when you do what. I was a bit suprised to learn that version 1.10 now used other dependencies (namely the CollectionUtils) than 1.9 did in the constructor call. Since there were dependency problems in 1.10 as well as in 1.9 I just sticked to the newer version.

I found the CollectionUtils located in the following artifact (as I was pointed there by its maven repository):

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-collections4</artifactId>
    <version>4.0</version>
</dependency>

Sadly that one (not obvious to me at first) doesn't define the class CollectionUtils in the package collections, but in the package collections4. It was hinted at this problem on the dependency overview, but they only mentioned possible problems with earlier versions... I appeared to be at a point of not thinking much about it anymore but simply changed the dependency to:

<dependency>
    <groupId>commons-collections</groupId>
    <artifactId>commons-collections</artifactId>
    <version>3.2.1</version>
</dependency>

I got everything to work (more or less, but the Exceptions I get now are not anymore depending on missing class definitions) after using these dependencies:

<dependencies>
    <dependency>
        <groupId>commons-configuration</groupId>
        <artifactId>commons-configuration</artifactId>
        <version>1.10</version>
    </dependency>
    <dependency>
        <groupId>commons-collections</groupId>
        <artifactId>commons-collections</artifactId>
        <version>3.2.1</version>
    </dependency>
    <dependency>
        <groupId>commons-beanutils</groupId>
        <artifactId>commons-beanutils</artifactId>
        <version>1.9.2</version>
    </dependency>
</dependencies>

Why do I have to add the dependencies myself? I thought the whole point in using maven is to avoid having to do such things and in terms of javadocs and source files it does a pretty good job.

By now I am convinced that the dependencies are not included in the hierarchy by design (is that so?), probably to avoid overhead. However is there a way to either simply get all dependencies at once or even better to get all dependencies I need? And why is it designed this way?

Answer 1

If we analyse commons-configuration's POM we see that the commons-collections dependency is optional:

  <dependencies>
    <dependency>
      <groupId>commons-collections</groupId>
      <artifactId>commons-collections</artifactId>
      <version>3.2.1</version>
      <optional>true</optional>
    </dependency>
    ...

Furthermore, from the Maven docs:

If a user wants to use functionality related to an optional dependency, they will have to redeclare that optional dependency in their own project.

Answer 2

This issue is explained on the Runtime dependencies page of the Commons Configuration website.

Quoting from that page:

A lot of dependencies are declared in the Maven POM. These are all needed during compile time. On runtime however you only need to add the dependencies to your classpath that are required by the parts of the Commons Configuration package you are using. The following table helps you to determine which dependencies you have to include based on the components you intend to use.

The other answers explain why this works from a Maven perspective. This answer is intended to provide a defence, of sorts, to the Commons Configuration folks. They did at least warn you!

In cases where the dependencies are on other Apache Commons components, they've taken the time to test with a variety of versions and have posted information on compatibility at the bottom of that page.

Answer 3

Maven tries to resolve all necessary dependencies for a library you're using in your pom. Well sometimes you have some dependencies which are only necessary for some specific features and you don't want to force the user of your dependency to download it if he doesn't use it. Then you're declaring your dependency as optional. This happened with commons-collections within commons-configuration. See commons-configuration-pom here