Jenkins Job Fails Sporadically with "Unexpected termination of the channel" on Remote Node

Question

I'm encountering intermittent failures with a Jenkins job running on a remote node. The job typically takes 2-3 hours to complete, but fails unpredictably with an "Unexpected termination of the channel" error.

Environment:

• Jenkins version: 2.462.3
• Remote node OS: SUSE Linux Enterprise Server 15 SP4
• Remote node JAVA version:

openjdk 17.0.14 2025-01-21
OpenJDK Runtime Environment (build 17.0.14+7-suse-150400.3.51.1-x8664)
OpenJDK 64-Bit Server VM (build 17.0.14+7-suse-150400.3.51.1-x8664, mixed mode, sharing)

Failure Pattern:

• Failures occur at random points during execution.
• Sometimes fails at job start, other times mid-execution.

Error Log (Build Log):

Started by timer
Running as SYSTEM
Building remotely on remote-node-xxx (worker-xxx) in workspace /data/jenkins/workspace/job-name
FATAL: java.io.IOException: Unexpected termination of the channel
java.io.EOFException
    at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
    at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
    at java.base/java.io.ObjectInputStream.readStreamHeader(Unknown Source)
    at java.base/java.io.ObjectInputStream.<init>(Unknown Source)
    at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:50)
    at hudson.remoting.Command.readFrom(Command.java:142)
    at hudson.remoting.Command.readFrom(Command.java:128)
    at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: java.io.IOException: Unexpected termination of the channel
    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to CCloud-de-i588324
        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1826)
        at hudson.remoting.Request.call(Request.java:199)
        at hudson.remoting.Channel.call(Channel.java:1041)
        at hudson.FilePath.act(FilePath.java:1229)
        at hudson.FilePath.act(FilePath.java:1218)
        at hudson.FilePath.mkdirs(FilePath.java:1409)
        at hudson.model.AbstractProject.checkout(AbstractProject.java:1241)
        at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:649)
        at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:521)
        at hudson.model.Run.execute(Run.java:1894)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
        at hudson.model.ResourceController.execute(ResourceController.java:101)
        at hudson.model.Executor.run(Executor.java:446)
Caused: hudson.remoting.RequestAbortedException
    at hudson.remoting.Request.abort(Request.java:346)
    at hudson.remoting.Channel.terminate(Channel.java:1122)
    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:90)
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Finished: FAILURE

What I've tried:

• Upgrade JAVA from 11 to 17
• Restart remote node
• Check sshd log but not helped (no error found)

Questions:

• What could cause these sporadic channel termination errors?
• How can I effectively debug the connection issues between Jenkins master and the remote node?
• Are there specific logs I should be monitoring on either the master or remote node to identify the root cause?

Any guidance on troubleshooting approaches or potential solutions would be greatly appreciated.