CODEWITHSUNDEEP
dockerjenkinsgradle
daniel purdew
daniel purdew

Reputation: 43

How to work around gradle being broken in docker environments?

Gradle does not work correctly in a docker environment, it is destined to use too much memory and be killed for using too much memory.

The memory manager gets its snapshots using the following class

https://github.com/gradle/gradle/blob/master/subprojects/process-services/src/main/java/org/gradle/process/internal/health/memory/MemInfoOsMemoryInfo.java

and in particular Gradle determines how much free memory is left by reading /proc/meminfo, which provides an inaccurate reading in a container.

Gradle only kills off Worker Daemons when a request comes in to make a new Worker Daemon with a larger min heap size then is available according to this reading.

Thus, Gradle will keep making workers until it uses up the alotted amount for the container and be killed.

Does anyone have a workaround for this? Don't really understand how this hasn't been a problem for more people. I suppose it only really becomes an issue if your worker daemons can't be reused and so new ones get created, which is the case for me as I have a large number of modules.

I have a temporary workaround wherein I give every jvm spawned a huge -Xms and so it always triggers the min heap size > available and so always removes prior worker daemons, but this is not satisfactory.

-- edit

To preempt some things, --max-workers does not affect the number of Worker Daemons allowed to exist, it merely affects the number which are allowed to be active. Even with --max-workers = 1, it is allowed to have arbitrary many idle Worker Daemons.

Upvotes: 3

Views: 534

Answers (1)

daniel purdew
daniel purdew

Reputation: 43

Edit - Ignore the below, it somewhat works but I have since patched Gradle by overwriting the MemInfoOsMemoryInfo class and it works a lot better. Will provide a link to the MR onto Gradle soon.

Found a reasonable work around, we listen for the os memory updates, and every time a task is done we request more memory than is determined to be free, ensuring a daemon is stopped.

import org.gradle.process.internal.health.memory.OsMemoryStatus
import org.gradle.process.internal.health.memory.OsMemoryStatusListener
import org.gradle.process.internal.health.memory.MemoryManagertask 

task expireWorkers {
    doFirst {
        long freeMemory = 0

        def memoryManager = services.get(MemoryManager.class)
        gradle.addListener(new TaskExecutionListener() {
            void beforeExecute(Task task) {
            }
            void afterExecute(Task task, TaskState state) {
                println "Freeing up memory"
                memoryManager.requestFreeMemory(freeMemory * 2)
            }
        })
        memoryManager.addListener(new OsMemoryStatusListener() {
            void onOsMemoryStatus(OsMemoryStatus osMemoryStatus) {
                freeMemory = osMemoryStatus.freePhysicalMemory
            }
        })
    }
}

Upvotes: 1

Related Questions