CODEWITHSUNDEEP
testingconsolegradle
tolitius
tolitius

Reputation: 22499

Gradle: How to Display Test Results in the Console in Real Time?

I would like to see test results ( system.out/err, log messages from components being tested ) as they run in the same console I run:

gradle test

And not wait until tests are done to look at the test reports ( that are only generated when tests are completed, so I can't "tail -f" anything while tests are running )

Upvotes: 350

Views: 247577

Answers (19)

Cafer Mert Ceyhan
Cafer Mert Ceyhan

Reputation: 1756

I've written a test logger for Kotlin DSL. You can put this block on your project scope build.gradle.kts file.

subprojects {
    tasks.withType(Test::class.java) {
        testLogging {
            showCauses = false
            showExceptions = false
            showStackTraces = false
            showStandardStreams = false

            val ansiReset = "\u001B[0m"
            val ansiGreen = "\u001B[32m"
            val ansiRed = "\u001B[31m"
            val ansiYellow = "\u001B[33m"

            fun getColoredResultType(resultType: ResultType): String {
                return when (resultType) {
                    ResultType.SUCCESS -> "$ansiGreen $resultType $ansiReset"
                    ResultType.FAILURE -> "$ansiRed $resultType $ansiReset"
                    ResultType.SKIPPED -> "$ansiYellow $resultType $ansiReset"
                }
            }

            afterTest(
                KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
                    println("${desc.className} | ${desc.displayName} = ${getColoredResultType(result.resultType)}")
                })
            )

            afterSuite(
                KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
                    if (desc.parent == null) {
                        println("Result: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)")
                    }
                })
            )
        }
    }
}

enter image description here

Upvotes: 3

Bruno Medeiros
Bruno Medeiros

Reputation: 2389

A more comprehensive response for those using th Kotlin DSL:

subprojects {
    // all the other stuff
    // ...
    tasks.named<Test>("test") {
        useJUnitPlatform()
        setupTestLogging()
    }
}

fun Test.setupTestLogging() {
    testLogging {
        events(
            org.gradle.api.tasks.testing.logging.TestLogEvent.FAILED,
            org.gradle.api.tasks.testing.logging.TestLogEvent.PASSED,
            org.gradle.api.tasks.testing.logging.TestLogEvent.SKIPPED,
            org.gradle.api.tasks.testing.logging.TestLogEvent.STANDARD_OUT,
        )
        exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
        showExceptions = true
        showCauses = true
        showStackTraces = true

        addTestListener(object : TestListener {
            override fun beforeSuite(suite: TestDescriptor) {}
            override fun beforeTest(testDescriptor: TestDescriptor) {}
            override fun afterTest(testDescriptor: TestDescriptor, result: TestResult) {}
            override fun afterSuite(suite: TestDescriptor, result: TestResult) {
                if (suite.parent != null) { // will match the outermost suite
                    val output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)"
                    val startItem = "|  "
                    val endItem = "  |"
                    val repeatLength = startItem.length + output.length + endItem.length
                    val messages = """
                        ${(1..repeatLength).joinToString("") { "-" }}
                        $startItem$output$endItem
                        ${(1..repeatLength).joinToString("") { "-" }}
                    """.trimIndent()
                    println(messages)
                }
            }
        })
    }
}

This should produce an output close to @odemolliens answers.

Upvotes: 2

Luis
Luis

Reputation: 3571

For Android, this works nicely:

android {
...

testOptions {
    unitTests.all {
        testLogging {
            outputs.upToDateWhen { false }
            events "passed", "failed", "skipped", "standardError"
            showCauses true
            showExceptions true
        }
    }
}

}

See Running Android unit / instrumentation tests from the console

Upvotes: 7

Mr-IDE
Mr-IDE

Reputation: 7641

Following on from Benjamin Muschko's answer (19 March 2011), you can use the -i flag along with grep, to filter out 1000s of unwanted lines. Examples:

Strong filter - Only display each unit test name and test result and the overall build status. Setup errors or exceptions are not displayed.

./gradlew test -i | grep -E " > |BUILD"

Soft filter - Display each unit test name and test result, as well as setup errors/exceptions. But it will also include some irrelevant info:

./gradlew test -i | grep -E -v "^Executing |^Creating |^Parsing |^Using |^Merging |^Download |^title=Compiling|^AAPT|^future=|^task=|:app:|V/InstrumentationResultParser:"

Soft filter, Alternative syntax: (search tokens are split into individual strings)

./gradlew test -i | grep -v -e "^Executing " -e "^Creating " -e "^Parsing " -e "^Using " -e "^Merging " -e "^Download " -e "^title=Compiling" -e "^AAPT" -e "^future=" -e "^task=" -e ":app:" -e "V/InstrumentationResultParser:"

Explanation of how it works:

The first command is ./gradlew test -i and "-i" means "Info/Verbose" mode, which prints the result of each test in real-time, but also displays large amounts of unwanted debug lines.

So the output of the first command, ./gradlew test -i, is piped to a second command grep, which will filter out many unwanted lines, based on a regular expression. "-E" enables the regular expression mode for a single string; "-e" enables regular expressions for multiple strings; and "|" in the regex string means "or".

In the strong filter, a unit test name and test result is allowed to display using " > ", and the overall status is allowed with "BUILD".

In the soft filter, the "-v" flag means "not containing" and "^" means "start of line". So it strips out all lines that start with "Executing " or start with "Creating ", etc.

Example for Android instrumentation unit tests, with gradle 5.1:

./gradlew connectedDebugAndroidTest --continue -i | grep -v -e \
    "^Transforming " -e "^Skipping " -e "^Cache " -e "^Performance " -e "^Creating " -e \
    "^Parsing " -e "^file " -e "ddms: " -e ":app:" -e "V/InstrumentationResultParser:"

Example for Jacoco unit test coverage, with gradle 4.10:

./gradlew createDebugCoverageReport --continue -i | grep -E -v "^Executing |^Creating |^Parsing |^Using |^Merging |^Download |^title=Compiling|^AAPT|^future=|^task=|:app:|V/InstrumentationResultParser:"

Upvotes: 3

tonisives
tonisives

Reputation: 2510

If you are using jupiter and none of the answers work, consider verifying it is setup correctly:

test {
    useJUnitPlatform()
    outputs.upToDateWhen { false }
}

dependencies {
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.7.0'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.7.0'
}

And then try the accepted answers

Upvotes: 2

Abhijit Sarkar
Abhijit Sarkar

Reputation: 24518

For those using Kotlin DSL, you can do:

tasks {
  named<Test>("test") {
    testLogging.showStandardStreams = true
  }
}

Upvotes: 3

Suleman
Suleman

Reputation: 41

Just add the following closure to your build.gradle. the output will be printed after the execution of every test.

test{
    useJUnitPlatform()
    afterTest { desc, result ->
        def output = "Class name: ${desc.className}, Test name: ${desc.name},  (Test status: ${result.resultType})"
        println( '\n' + output)
    }
}

Upvotes: 4

Enrico
Enrico

Reputation: 230

If you have a build.gradle.kts written in Kotlin DSL you can print test results with (I was developing a kotlin multi-platform project, with no "java" plugin applied):

tasks.withType<AbstractTestTask> {
    afterSuite(KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
        if (desc.parent == null) { // will match the outermost suite
            println("Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)")
        }
    }))
}

Upvotes: 7

adarshr
adarshr

Reputation: 62573

Disclaimer: I am the developer of the Gradle Test Logger Plugin.

You can simply use the Gradle Test Logger Plugin to print beautiful logs on the console. The plugin uses sensible defaults to satisfy most users with little or no configuration but also offers a number of themes and configuration options to suit everyone.

Examples

Standard Theme Standard theme

Mocha Theme Mocha theme

Usage

plugins {
    id 'com.adarshr.test-logger' version '<version>'
}

Make sure you always get the latest version from Gradle Central.

Configuration

You don't need any configuration at all. However, the plugin offers a few options. This can be done as follows (default values shown):

testlogger {
    // pick a theme - mocha, standard, plain, mocha-parallel, standard-parallel or plain-parallel
    theme 'standard'

    // set to false to disable detailed failure logs
    showExceptions true

    // set to false to hide stack traces
    showStackTraces true

    // set to true to remove any filtering applied to stack traces
    showFullStackTraces false

    // set to false to hide exception causes
    showCauses true

    // set threshold in milliseconds to highlight slow tests
    slowThreshold 2000

    // displays a breakdown of passes, failures and skips along with total duration
    showSummary true

    // set to true to see simple class names
    showSimpleNames false

    // set to false to hide passed tests
    showPassed true

    // set to false to hide skipped tests
    showSkipped true

    // set to false to hide failed tests
    showFailed true

    // enable to see standard out and error streams inline with the test results
    showStandardStreams false

    // set to false to hide passed standard out and error streams
    showPassedStandardStreams true

    // set to false to hide skipped standard out and error streams
    showSkippedStandardStreams true

    // set to false to hide failed standard out and error streams
    showFailedStandardStreams true
}

I hope you will enjoy using it.

Upvotes: 250

Shubham Chaudhary
Shubham Chaudhary

Reputation: 50963

Here is my fancy version:

fancy test result

import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent

tasks.withType(Test) {
    testLogging {
        // set options for log level LIFECYCLE
        events TestLogEvent.FAILED,
               TestLogEvent.PASSED,
               TestLogEvent.SKIPPED,
               TestLogEvent.STANDARD_OUT
        exceptionFormat TestExceptionFormat.FULL
        showExceptions true
        showCauses true
        showStackTraces true

        // set options for log level DEBUG and INFO
        debug {
            events TestLogEvent.STARTED,
                   TestLogEvent.FAILED,
                   TestLogEvent.PASSED,
                   TestLogEvent.SKIPPED,
                   TestLogEvent.STANDARD_ERROR,
                   TestLogEvent.STANDARD_OUT
            exceptionFormat TestExceptionFormat.FULL
        }
        info.events = debug.events
        info.exceptionFormat = debug.exceptionFormat

        afterSuite { desc, result ->
            if (!desc.parent) { // will match the outermost suite
                def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)"
                def startItem = '|  ', endItem = '  |'
                def repeatLength = startItem.length() + output.length() + endItem.length()
                println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
            }
        }
    }
}

Upvotes: 260

stefanglase
stefanglase

Reputation: 10402

You can add a Groovy closure inside your build.gradle file that does the logging for you:

test {
    afterTest { desc, result -> 
        logger.quiet "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
    }
}

On your console it then reads like this:

:compileJava UP-TO-DATE
:compileGroovy
:processResources
:classes
:jar
:assemble
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
Executing test maturesShouldBeCharged11DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test studentsShouldBeCharged8DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test seniorsShouldBeCharged6DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test childrenShouldBeCharged5DollarsAnd50CentForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
:check
:build

Since version 1.1 Gradle supports much more options to log test output. With those options at hand you can achieve a similar output with the following configuration:

test {
    testLogging {
        events "passed", "skipped", "failed"
    }
}

Upvotes: 179

Andrzej Rehmann
Andrzej Rehmann

Reputation: 13790

My favourite minimalistic version based on Shubham Chaudhary answer. enter image description here

Put this in build.gradle file:

test {
    afterSuite { desc, result ->
    if (!desc.parent)
        println("${result.resultType} " +
            "(${result.testCount} tests, " +
            "${result.successfulTestCount} successes, " +
            "${result.failedTestCount} failures, " +
            "${result.skippedTestCount} skipped)")
    }
}

Upvotes: 20

odemolliens
odemolliens

Reputation: 2641

Merge of Shubham's great answer and JJD use enum instead of string

tasks.withType(Test) {
   testLogging {
       // set options for log level LIFECYCLE
       events TestLogEvent.PASSED,
            TestLogEvent.SKIPPED, TestLogEvent.FAILED, TestLogEvent.STANDARD_OUT
       showExceptions true
       exceptionFormat TestExceptionFormat.FULL
       showCauses true
       showStackTraces true

    // set options for log level DEBUG and INFO
       debug {
        events TestLogEvent.STARTED, TestLogEvent.PASSED, TestLogEvent.SKIPPED, TestLogEvent.FAILED, TestLogEvent.STANDARD_OUT, TestLogEvent.STANDARD_ERROR
        exceptionFormat TestExceptionFormat.FULL
       }
       info.events = debug.events
       info.exceptionFormat = debug.exceptionFormat

       afterSuite { desc, result ->
           if (!desc.parent) { // will match the outermost suite
               def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
               def startItem = '|  ', endItem = '  |'
               def repeatLength = startItem.length() + output.length() + endItem.length()
               println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
           }
       }
   }
}

Upvotes: 3

JJD
JJD

Reputation: 51804

As a follow up to Shubham's great answer I like to suggest using enum values instead of strings. Please take a look at the documentation of the TestLogging class.

import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent

tasks.withType(Test) {
    testLogging {
        events TestLogEvent.FAILED,
               TestLogEvent.PASSED,
               TestLogEvent.SKIPPED,
               TestLogEvent.STANDARD_ERROR,
               TestLogEvent.STANDARD_OUT
        exceptionFormat TestExceptionFormat.FULL
        showCauses true
        showExceptions true
        showStackTraces true
    }
}

Upvotes: 26

Langusten Gustel
Langusten Gustel

Reputation: 10992

As stefanglase answered:

adding the following code to your build.gradle (since version 1.1) works fine for output on passed, skipped and failed tests.

test {
    testLogging {
        events "passed", "skipped", "failed", "standardOut", "standardError"
    }
}

What I want to say additionally (I found out this is a problem for starters) is that the gradle test command executes the test only one time per change.

So if you are running it the second time there will be no output on test results. You can also see this in the building output: gradle then says UP-TO-DATE on tests. So its not executed a n-th time.

Smart gradle!

If you want to force the test cases to run, use gradle cleanTest test.

This is slightly off topic but I hope it will help some newbies.

edit

As sparc_spread stated in the comments:

If you want to force gradle to always run fresh tests (which might not always be a good idea) you can add outputs.upToDateWhen {false} to testLogging { [...] }. Continue reading here.

Peace.

Upvotes: 160

Darwin
Darwin

Reputation: 4786

Add this to build.gradle to stop gradle from swallowing stdout and stderr.

test {
    testLogging.showStandardStreams = true
}

It's documented here.

Upvotes: 72

user3521637
user3521637

Reputation: 1652

'test' task does not work for Android plugin, for Android plugin use the following:

// Test Logging
tasks.withType(Test) {
    testLogging {
        events "started", "passed", "skipped", "failed"
    }
}

See the following: https://stackoverflow.com/a/31665341/3521637

Upvotes: 47

radeklos
radeklos

Reputation: 2248

In Gradle using Android plugin:

gradle.projectsEvaluated {
    tasks.withType(Test) { task ->
        task.afterTest { desc, result ->
            println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
        }
    }
}

Then the output will be:

Executing test testConversionMinutes [org.example.app.test.DurationTest] with result: SUCCESS

Upvotes: 7

Benjamin Muschko
Benjamin Muschko

Reputation: 33426

You could run Gradle with INFO logging level on the command line. It'll show you the result of each test while they are running. Downside is that you will get far more output for other tasks also.

gradle test -i

Upvotes: 258

Related Questions