CODEWITHSUNDEEP
antjunitteamcityjacoco
user2100163
user2100163

Reputation: 91

Failed junit test not catched by teamcity when using jacoco

I was trying to integrate code coverage on my project by using jacoco, ant and teamcity. However, I realized that when jacoco task is around the junit task, teamcity does not catch the failing tests and everything is a success even with test failed.

Here are my 2 test tasks to test with and without jacoco and see teamcity bahaviours.

1- with jacoco activated

<target name="-test">
    <echo message="JaCoCo activated"/>
    <!-- Import the JaCoCo Ant Task -->
    <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml"/>
    <!-- Run your unit tests, adding the JaCoCo agent -->
    <jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
        <junit fork="yes" printsummary="yes" haltonfailure="no" showoutput="false" failureProperty="test.failed" errorProperty="test.failed">
            <classpath>
                <path location="${lib}/${projectName}.jar"/>
                <path refid="project.classpath"/>
            </classpath>
            <formatter type="xml"/>
            <batchtest todir="${reportingHome}">
                <fileset dir="${test}">
                    <include name="**/*Test.java"/>
                </fileset>
            </batchtest>
        </junit>
    </jacoco:coverage>
    <copy todir="${completeReportDir}" overwrite="true">
        <fileset dir="${reportingHome}">
            <include name="*.xml"/>
        </fileset>
    </copy>
</target>

2- without jacoco

<target name="-test">
    <echo message="JaCoCo activated"/>
    <!-- Import the JaCoCo Ant Task -->
    <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml"/>
    <!-- Run your unit tests, adding the JaCoCo agent -->
    <!--<jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">-->
        <junit fork="yes" printsummary="yes" haltonfailure="no" showoutput="false" failureProperty="test.failed" errorProperty="test.failed">
            <classpath>
                <path location="${lib}/${projectName}.jar"/>
                <path refid="project.classpath"/>
            </classpath>
            <formatter type="xml"/>
            <batchtest todir="${reportingHome}">
                <fileset dir="${test}">
                    <include name="**/*Test.java"/>
                </fileset>
            </batchtest>
        </junit>
    <!--</jacoco:coverage>-->
    <copy todir="${completeReportDir}" overwrite="true">
        <fileset dir="${reportingHome}">
            <include name="*.xml"/>
        </fileset>
    </copy>
</target>

Only jacoco task has been commented between the 2 releases of test. Teamcity output 

[CommonBuildTasks.-test] echo
[08:26:21]: [echo] JaCoCo activated
[08:26:21]: [CommonBuildTasks.-test] jacoco:coverage (4s)
[08:26:21]: [jacoco:coverage] Enhancing junit with coverage.
[08:26:22]: [jacoco:coverage] Running ca.thalesgroup.socialnetworkanalysisorchestrator.impl.client.SocialNetworkAnalysisOrchestratorServiceProviderTest
[08:26:25]: [jacoco:coverage] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 3.511 sec
[08:26:26]: [jacoco:coverage] Test ca.thalesgroup.socialnetworkanalysisorchestrator.impl.client.SocialNetworkAnalysisOrchestratorServiceProviderTest FAILED
[08:26:26]: [CommonBuildTasks.-test] copy
[08:26:26]: [copy] Copying 1 file to C:\TeamCity\buildAgent\work\cc10e09e43249f57\reports

As you can see, a test failed but teamcity has reported a successfull build.

Any idea why I got this behaviour? Thanks

Upvotes: 3

Views: 2167

Answers (2)

Max Shcherbakov
Max Shcherbakov

Reputation: 11

I solved this issue by using agent task instead of the coverage task. So, instead of 

<jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">

Use:

<jacoco:agent property="agentvmparam" destfile="${bin}/jacoco.exec"/>
<junit fork="yes"...
    <jvmarg value="${agentvmparam}"/>
</junit>

Agent task uses the same properties as the coverage task. Then you can start your junit task without wrapping it in coverage task. That way teamcity is able to intercept junit task output.

Upvotes: 1

Markus
Markus

Reputation: 2053

The answer is hidden in your call to the JUnit-Task:

<junit haltonfailure="no">...</junit>

With this configuration, the JUnit task does not fail the build on failing tests. This should lead to the desired behaviour:

<junit haltonfailure="yes">...</junit>

See the Ant documentation for the configuration of the JUnit task.

Upvotes: 1

Related Questions