CODEWITHSUNDEEP
javatestcontainers
Alexander Argyriou
Alexander Argyriou

Reputation: 411

Cannot connect to Ryuk

As far as i know, ryuk serves as the main independent resource handler container for other testcontainers.

I have observed unexpected behavior during the execution of integration tests in both the local environment and Jenkins using a Postgres testcontainer. Occasionally (though infrequently), I encounter the following warning message:

WARN org.testcontainers.utility.ResourceReaper - Can not connect to Ryuk at localhost:32769 java.net.SocketException: Connection reset

followed by this error

07:59:16.503 [main] ERROR org.testcontainers.utility.ResourceReaper - Timed out waiting for Ryuk container to start. Ryuk's logs:
2023/06/09 11:58:46 Pinging Docker...
2023/06/09 11:58:46 Docker daemon is available!
2023/06/09 11:58:46 Starting on port 8080...
2023/06/09 11:58:46 Started!

Once the Docker service is restarted or Ryuk is disabled, the situation returns to normal. Alternatively, the issue may resolve itself after several attempts.

I have read several threads such as

https://github.com/testcontainers/testcontainers-java/issues/3609

https://github.com/testcontainers/testcontainers-java/issues/4954

https://jira.atlassian.com/browse/BCLOUD-17236

https://github.com/testcontainers/testcontainers-java/issues/581

However, I have not identified any specific reasons explaining this behavior. Also i am a bit confused with the provided discussions, answers and closed threads without any specific solution point.

Testcontainers version is 1.15.1. Error occured on ubuntu 20.04 and Amazon linux.

Has anyone encountered a similar issue? If so, what was determined to be the cause? Additionally, are there any alternatives to Ryuk for resource reaping purposes?

Upvotes: 3

Views: 5948

Answers (1)

Andrey B. Panfilov
Andrey B. Panfilov

Reputation: 6083

I believe your "ryuk issue" is not directly related to testcontainers functionality, however there is a room for improvement, at least for linux hosts, the reasoning is following:

your log

07:59:16.503 [main] ERROR org.testcontainers.utility.ResourceReaper - Timed out waiting for Ryuk container to start. Ryuk's logs:
2023/06/09 11:58:46 Pinging Docker...
2023/06/09 11:58:46 Docker daemon is available!
2023/06/09 11:58:46 Starting on port 8080...
2023/06/09 11:58:46 Started!

reveals following:

  • ryuk container got started
  • ResourceReaper has been trying to reach it during 30 seconds and failed

under "normal conditions" ryuk v0.3.0 logs should look like:

2023/06/10 15:15:45 Pinging Docker...
2023/06/10 15:15:45 Docker daemon is available!
2023/06/10 15:15:45 Starting on port 8080...
2023/06/10 15:15:45 Started!
2023/06/10 15:15:48 Connected

i.e. ryuk container typically logs information about clients connected (last line in my listing), but your log does not contain such information, which in turn could mean following:

  • operating system (or java) was to busy to schedule execution of spawned thread
  • logging engine or ResourceReaper is broken in testcontainers v1.15.1 and it does not log enough diagnostic information (anyway, 1.15.1 version is definitely too old and it is better off updating testcontainers first)
  • there is some network connectivity issue caused by one of the following: overlapping networks, firewall setup, ephemeral port ranges, issues in docker engine.

Upvotes: 1

Related Questions