Maven building jar: merge duplicate resources from dependencies

Question

I'm getting this problem:

I have a project with spring-boot. It works perfectly on eclipse, but I need to generate a jar with dependencies, so I add the maven-assembly-plugin configuration to the pom.

Some spring dependencies has in the META-INF a file called spring.schemas, and I need to merge all spring.schemas into one (spring-context, spring-beans, etc)

I tried this solution using maven-shade-pluggin and AppendingTransformer, and it merges all spring.schemas perfectly... But it has an issue, when I execute the jar, it fails with:

java.lang.IllegalStateException: Unable to open nested entry 'lib/spring-boot-starter-batch-1.2.4.RELEASE.jar'. 
It has been compressed and nested jar files must be stored without compression.
Please check the mechanism used to create your executable jar file

So, shade plugin compress the jar, and spring-boot doesn't like it, and there's no way to turn off the compression in shade. I manually copy the shade spring.schemas generated by shade and I put it in the maven-assembly-pluggin generated and uncompressed jar with dependencies. It works.

Then I tried to include the generated spring.schemas into my resources folder, but it's allways overrided by the spring.schemas from spring-context.

I also tried this other solution to exclude spring.schemas from dependency jar using a descriptor XML in assembly, but it doesn't work:

<assembly
    xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">
    <id>distribution</id>
    <formats>
        <format>jar</format>
    </formats>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <includes>
                <include>org.springframework:spring-context</include>
            </includes>
            <unpack>true</unpack>
            <unpackOptions>
                <excludes>
                    <exclude>**/spring.schemas</exclude>
                </excludes>
            </unpackOptions>
        </dependencySet>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <excludes>
                <exclude>org.springframework:spring-context</exclude>
            </excludes>
            <unpack>true</unpack>
        </dependencySet> 
    </dependencySets>
</assembly>

            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                    <archive>
                        <manifest>
                            <mainClass>org.springframework.batch.core.launch.support.CommandLineJobRunner</mainClass>
                        </manifest>
                        <compress>false</compress>
                    </archive>
                    <descriptors>
                        <descriptor>src/main/assembly/distribution.xml</descriptor>
                    </descriptors>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

This are my dependencies:

<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
        </dependency>
        <!-- MySql 5.5 Connector -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.35</version>
        </dependency>
    </dependencies>

Any ideas? Thanks in advance

Answer 1

You can use maven-assembly-plugin.

Add the plugin to your pom.xml:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-assembly-plugin</artifactId>
  <version>3.1.1</version>
  <executions>
    <execution>
      <id>make-assembly</id>
      <phase>package</phase>
      <goals>
        <goal>single</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <descriptors>
      <descriptor>fat-jar.xml</descriptor>
    </descriptors>
  </configuration>
</plugin>

Add a fat-jar.xml descriptor next to your pom.xml:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
  <id>fat</id>
  <formats>
    <format>jar</format>
  </formats>
  <includeBaseDirectory>false</includeBaseDirectory>
  <dependencySets>
    <dependencySet>
      <outputDirectory>/</outputDirectory>
      <useProjectArtifact>true</useProjectArtifact>
      <unpack>true</unpack>
      <scope>runtime</scope>
    </dependencySet>
  </dependencySets>
  <containerDescriptorHandlers>
    <containerDescriptorHandler>
      <handlerName>metaInf-spring</handlerName>
    </containerDescriptorHandler>
  </containerDescriptorHandlers>
</assembly>

Notice the metaInf-spring handler. It will merge any META-INF duplicate files having names starting with the spring word.

The reference documentation: https://maven.apache.org/plugins/maven-assembly-plugin/examples/single/using-container-descriptor-handlers.html

Answer 2

Instead of using maven-assembly-plugin and maven-shade-pluggin, I would try to use Spring Boot's own spring-boot-maven-plugin that can be used to create an executable ‘fat’ JAR. See this reference documentation page, section "73.2 Create an executable JAR with Maven".

<build>
<plugins>
    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
</plugins>

Or, if you don't use the spring-boot-starter-parent POM:

<build>
<plugins>
    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>1.2.5.RELEASE</version>
        <executions>
            <execution>
                <goals>
                    <goal>repackage</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>