CODEWITHSUNDEEP
javagradle
Zombie
Zombie

Reputation: 1

Gradle task fails intermittently

While running a gradle task it fails intermittently (1-2 out of 10 times on avg which does not makes sense)

task runIntProxy(dependsOn: [':int-proxy:getVersions',':infrastructure:dev:startNginx']) {
  group 'start'
  description 'Starts int-proxy'
    doLast {
        file(logsDir).mkdirs()
        def imageTag = readVersion('int_proxy_image_tag')

        command("docker-compose ${composeFiles} up -d", ['env': ['INT_PROXY_IMAGE_TAG': imageTag]])
        command("docker-compose ${composeFiles} logs -f", ['printOutput': false, 'waitForExit': false, 'outputFile': "$logsDir/int-proxy.log"])

        quiet "Running healthchecks:"
        println(intProxyStatus)
        try {
        def response = new URL(intProxyStatus).getContent().text
        println("output from URL:" + response)
      
        if(response.contains("alive")){
          quiet("Int-proxy is up and running")
        } else {
          throw new GradleException("Int-proxy is not up!")
        }
        } catch (GradleException e) { }
    }
}

since it was failing falsely (while performing same task manually it works fine but with jenkins it fails) I added catch (GradleException e) { } but still it fails intermittently can anyone point out what am I missing?

error code is below

* What went wrong:
Execution failed for task ':int-proxy:runIntProxy'.
> java.net.SocketException: Connection reset

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':int-proxy:runIntProxy'.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:121)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:117)
    at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:184)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:110)
    at org.gradle.api.internal.tasks.execution.ResolveIncrementalChangesTaskExecuter.execute(ResolveIncrementalChangesTaskExecuter.java:84)
    at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:92)
    at org.gradle.api.internal.tasks.execution.FinishSnapshotTaskInputsBuildOperationTaskExecuter.execute(FinishSnapshotTaskInputsBuildOperationTaskExecuter.java:51)
    at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:103)
    at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:110)
    at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
    at org.gradle.api.internal.tasks.execution.StartSnapshotTaskInputsBuildOperationTaskExecuter.execute(StartSnapshotTaskInputsBuildOperationTaskExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
    at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
    at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:63)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:49)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:46)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
    at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.api.UncheckedIOException: java.net.SocketException: Connection reset
    at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:61)
    at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:41)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:367)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:326)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:311)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$200(ExecuteActionsTaskExecuter.java:75)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:158)
    at org.gradle.internal.execution.impl.steps.ExecuteStep.execute(ExecuteStep.java:46)
    at org.gradle.internal.execution.impl.steps.CancelExecutionStep.execute(CancelExecutionStep.java:34)
    at org.gradle.internal.execution.impl.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:69)
    at org.gradle.internal.execution.impl.steps.TimeoutStep.execute(TimeoutStep.java:49)
    at org.gradle.internal.execution.impl.steps.CatchExceptionStep.execute(CatchExceptionStep.java:34)
    at org.gradle.internal.execution.impl.steps.CreateOutputsStep.execute(CreateOutputsStep.java:49)
    at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:42)
    at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:28)
    at org.gradle.internal.execution.impl.steps.CacheStep.executeWithoutCache(CacheStep.java:136)
    at org.gradle.internal.execution.impl.steps.CacheStep.lambda$execute$5(CacheStep.java:86)
    at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:85)
    at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:37)
    at org.gradle.internal.execution.impl.steps.PrepareCachingStep.execute(PrepareCachingStep.java:33)
    at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:38)
    at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:23)
    at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:95)
    at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:88)
    at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:52)
    at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:36)
    at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:34)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:109)
    ... 40 more
Caused by: java.net.SocketException: Connection reset
    at java_net_URL$getContent$0.call(Unknown Source)
    at build_4hqoiddv5luerpje0uiotdsrg$_run_closure4$_closure14.doCall(/home/jenkins/workspace/development/int-proxy/build.gradle:63)
    at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:644)
    at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:617)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:337)
    ... 72 more

also we poll to see if service is up or not

if(!waitForNoConnection(intProxyStatus, 5, 5, 15)) {
      throw new GradleException("Couldn't stop int-proxy. You might need to forcefully kill it yourself.")
    }

Upvotes: 0

Views: 159

Answers (2)

Zombie
Zombie

Reputation: 1

The issue was container was coming up but somehow service was not coming up and was taking some time, added a sleep of 15sec and it all started to go fine.

Upvotes: 0

jurez
jurez

Reputation: 4657

The root cause is java.net.SocketException: Connection reset.

This means that there was something wrong with the network:

  • Maybe the server is not running or reachable
  • Maybe you are trying to connect to the wrong IP/hostname
  • Maybe the server closed the connection because of some internal error
  • Maybe the transfer was too slow and connection timed out
  • Maybe you have a shitty firewall, etc.

Have you tried it enough times manually to be 99.9% sure that it is not also happening manually? Have you tried it on the same machine, with the same version of Java, Gradle and everything else? Have you made sure that the issue is not timing- or speed-related?

My initial thought was that you are starting a docker container, after which you make a URL request to the service, but you are forgetting that container start is asynchronous. In other words, you have no guarantee that the service will actually be running when docker-compose finishes. But this is just a guess.

Try adding a delay before making a URL request to see if it makes any change. The proper way to do it would be to poll the URL in a loop until the service either comes up or times out.

Upvotes: 1

Related Questions