"java.lang.IllegalArgumentException: already added" when building a project which depends on library projects

Question

Given

Commons - simple java project
AndroidLibrary1 - android library
AndroidLibrary2 - android library
AndroidProject - android project

And the projects have such references to each other:

AndroidLibrary1 -> Commons
AndroidLibrary2 -> Commons
AndroidProject -> AndroidLibrary1, AndroidLibrary2

Problem

When I build AndroidProject I get such error:

java.lang.IllegalArgumentException: already added: Lcom/test/Bar;

where "com.test.Bar" is a class from "Commons" project, that used by both AndroidLibrary1 and AndroidLibrary2.

Environment

Eclipse 3.7.1
android-sdk-windows_r15

Any ideas how to fix this?

EDIT: Found related discussion.

Answer 1

After considerable amount of time and regex black magic I have found this solution:

<target name="-post-compile">
    <macrodef name="dex-helper">
        <element name="external-libs" optional="yes" />
        <attribute name="nolocals" default="false" />
        <sequential>
            <!-- sets the primary input for dex. If a pre-dex task sets it to
                something else this has no effect -->
            <property name="out.dex.input.absolute.dir" value="${out.classes.absolute.dir}" />

            <!-- set the secondary dx input: the project (and library) jar files
                If a pre-dex task sets it to something else this has no effect -->
            <if>
                <condition>
                    <isreference refid="out.dex.jar.input.ref" />
                </condition>
                <else>
                    <path id="out.dex.jar.input.ref">
                        <path refid="jar.libs.ref" />
                    </path>
                </else>
            </if>

            <if>
                <condition>
                    <length string="${toString:out.dex.jar.input.ref}" trim="true" when="greater" length="0"/>
                </condition>
                <then>
                    <echo message="${toString:out.dex.jar.input.ref}" file="antler.tmp" />
                    <loadfile property="out.dex.jar.input.ref.fixed" srcFile="antler.tmp">
                        <filterchain>
                            <tokenfilter>
                                <replaceregex
                                    pattern="(?&lt;=;|^)[^;]+\\libs\\([^\\;]+)(?:;|$)(?=(?:(?&lt;=;|^)[^;]+(?:;|$))*(?&lt;=;|^)[^;]+\\\1(?:;|$))"
                                    replace="" flags="g"/>
                            </tokenfilter>
                        </filterchain>
                    </loadfile>
                    <delete file="antler.tmp"/>
                    <path id="out.dex.jar.input.ref.fixed" path="${out.dex.jar.input.ref.fixed}"/>
                </then>
                <else>
                    <path id="out.dex.jar.input.ref.fixed" />
                </else>
            </if>

            <dex executable="${dx}"
                    output="${intermediate.dex.file}"
                    nolocals="@{nolocals}"
                    verbose="${verbose}">
                <path path="${out.dex.input.absolute.dir}"/>
                <path refid="out.dex.jar.input.ref.fixed" />
                <external-libs />
            </dex>
        </sequential>
    </macrodef>
</target>

This is what you should add to your build.xml file before the <import file="${sdk.dir}/tools/ant/build.xml" /> line. In the case you have the same JAR in the libs folder in several of your projects, this code will remove the duplicate entries from the build system based on the file names.

Now, how this works

I will use the word "library" for JAR in libs folders and "library project" for the Android library projects on which other projects may depend. The problem is that when Android system builds projects with library projects, it just bundles all the libraries into one heap ignoring the fact that there may be duplicates. So we have to fix the paths.

So when the build gets to the point where the compilation is complete, we redefine the part which is responsible for converting Java code into DEX. The code above is exactly as in the Android build tools, except for the following part:

<if>
    <condition>
        <length string="${toString:out.dex.jar.input.ref}" trim="true" when="greater" length="0"/>
    </condition>
    <then>
        <echo message="${toString:out.dex.jar.input.ref}" file="antler.tmp" />
        <loadfile property="out.dex.jar.input.ref.fixed" srcFile="antler.tmp">
            <filterchain>
                <tokenfilter>
                    <replaceregex
                        pattern="(?&lt;=;|^)[^;]+\\libs\\([^\\;]+)(?:;|$)(?=(?:(?&lt;=;|^)[^;]+(?:;|$))*(?&lt;=;|^)[^;]+\\\1(?:;|$))"
                        replace="" flags="g"/>
                </tokenfilter>
            </filterchain>
        </loadfile>
        <delete file="antler.tmp"/>
        <path id="out.dex.jar.input.ref.fixed" path="${out.dex.jar.input.ref.fixed}"/>
    </then>
    <else>
        <path id="out.dex.jar.input.ref.fixed" />
    </else>
</if>

We have the unclean paths in out.dex.jar.input.ref, and our goal is to create paths without the duplicate entries (they will be stored in out.dex.jar.input.ref.fixed). Unfortunately we have to unload the path to a file, change it, and load it back, because ant for some weird reasons can't work with regex in properties (unless you are using external libraries like ant-contrib).

The regex which does the trick is (?<=;|^)[^;]+\\libs\\([^\\;]+)(?:;|$)(?=(?:(?<=;|^)[^;]+(?:;|$))*(?<=;|^)[^;]+\\\1(?:;|$)). This horrible mess of symbols matches the path which ends with libs\<something.jar> and has somewhere further another path which ends with <something.jar>. My regex uses Windows separator, so if you are under Linux or Mac OS X, you should change those \\ to the necessary separator or make the regex universal. Sorry, I was just in no mood to fix it having spent too much time on it already. If you want to check out how the regex works, you can do so on Regexr.

After we have removed the duplicates, we just have to load back the string to a new path, out.dex.jar.input.ref.fixed and use it to run dex. Hope my solution helps.