How to work around javac incremental build in the Android Tools not detecting changes that should cause recompilation?

Question

My coworkers and I recently upgraded our Android Tools to r19 and are seeing problems with the new incremental builds system Google introduced. Incremental compilation seems like a hard problem and it looks like they did not solve it so we are getting broken APKs now if we aren't careful.

I reported this issue to Google but I wanted to know if any one else has seen it and if so maybe they can lobby Google to fix it so engineering man-hours aren't looking at builds that are broken due to a bad build script.

Take a look here: https://code.google.com/p/android/issues/detail?id=31242

As a workaround I modified the Ant script to delete all the class files before the -build-setup step in Google's Ant script.

Answer 1

The issue described in that bug is actually a long standing limitation of most of java compilers and tools, related to .class files not maintaining enough information to allow tools to properly calculate dependencies. Javac doesn't even go that far, and instead only recompiles when the java file is newer than the corresponding .class file. That's by design, even.

The only sure answer I can find is to delete all your class files, every time you build, making your builds not incremental. You could do this by adding this to your build.xml:

<target name="-pre-compile" depends="-pre-compile-delete"/>

<target name="-pre-compile-delete">
    <!-- Workaround for https://code.google.com/p/android/issues/detail?id=31242 -->
    <do-only-if-manifest-hasCode elseText="hasCode = false. Skipping...">
        <delete failonerror="false" dir="${out.classes.absolute.dir}"/>
    </do-only-if-manifest-hasCode>
</target>

The JDK's built-in tool to address this is the depend task. It tries to traverse the class files, and delete files whose transitive dependencies have changed. However, as noted in the documentation, depend has some limitations all stemming again from class files not containing enough information to provided a full dependency tree.

Instead, I'd suggest a simpler but reasonably robust workaround using the dependset task. Instead of trying to get minimal incremental compilation using depend task, we delete all class files if any of the sources/jars are newer. We may recompile when it is actually not needed, however builds with no changes are still faster and we don't inadvertently skip compilation in cases where it is needed.

<target name="-pre-compile" depends="-pre-compile-dependset" />
<target name="-pre-compile-dependset">
    <!-- Alternative workaround for https://code.google.com/p/android/issues/detail?id=31242 -->
    <echo>Deleting all class files if newer sources (java or jars) are found...</echo>
    <do-only-if-manifest-hasCode elseText="hasCode = false. Skipping...">

        <!-- The names of these properties suggest they are single directories, but -->
        <!-- some projects with Android.mk files treat them like paths (with multiple dirs). -->
        <!-- Massage these paths get all the files inside multiple dirs. -->
        <path id="project.all.sources.path">
            <pathelement path="${source.absolute.dir}"/>
            <pathelement path="${gen.absolute.dir}"/>
        </path>

        <!-- convert project.all.sources.path to project.all.sources.list  -->
        <pathconvert
            refid="project.all.sources.path"
            property="project.all.sources.list"
            pathsep=","
            dirsep="/">

            <!-- Make the path elements relative to basedir and glob to match all files -->
            <!-- The result will look like: "src/**,gen/**" -->
            <chainedmapper>
                <filtermapper>
                    <replacestring from="${basedir}/" to=""/>
                </filtermapper>
                <regexpmapper from="^(.*)" to="\1/**"/>
            </chainedmapper>
        </pathconvert>

        <dependset verbose="true">
            <sources>
                    <!-- merge the project's own classpath and the tested project's classpath -->
                    <path refid="project.all.jars.path" />
                    <path refid="tested.project.classpath" />
                    <!-- All source files -->
                    <fileset dir="${basedir}" includes="${project.all.sources.list}" />
            </sources>
            <targets>
                <fileset dir="${out.classes.absolute.dir}"/>
            </targets>
        </dependset>
    </do-only-if-manifest-hasCode>
</target>

This solution is reliable enough that it I believe it could be incorporated in the the sdk build eventually.