Testcontainers, Docker in Docker with custom network, Containers don't belong to network

Question

I'm trying to get Testcontainers running on TeamCity using a Docker builder image.

The test runs fine locally (not inside the builder image). And only partially within the builder image on TeamCity. I followed the guide on DinD but there are no examples on how a docker network comes into play.

The way we start the build in TeamCity (note the --network param, ryuk is disabled as it had connection issues):

docker network create --driver bridge custom_network

docker run --rm -it -v $PWD:$PWD -w $PWD \
  --privileged \
  --network=custom_network \
  -e TESTCONTAINERS_RYUK_DISABLED=true \
  -e _JAVA_OPTIONS="" \
  -e DOCKER_HOST="unix:///var/run/docker.sock" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /home/teamcity/.docker:/home/java/.docker
  -v /local/maven/cache/repository:/opt/m2/repository \
  registry.ch/java:11-builder \
  mvn verify

The build runs quite normal: the junit test starts, the custom oracle-xe image we use is downloaded, and the log suggests it is started. But locally I can see that testcontainers is polling to create a connection, on TeamCity the build just continues and runs into an error:

[14:01:07] :     [Step 3/3] 14:01:07.006 [tc-okhttp-stream-276714561] DEBUG com.github.dockerjava.core.command.PullImageResultCallback - ResponseItem(stream=null, status=Extracting, progressDetail=ResponseItem.ProgressDetail(current=625569807, total=625569807, start=null), progress=[==================================================>]  625.6MB/625.6MB, id=2538d1d7e815, from=null, time=null, errorDetail=null, error=null, aux=null)
[14:01:07] :     [Step 3/3] 14:01:07.211 [tc-okhttp-stream-276714561] DEBUG com.github.dockerjava.core.command.PullImageResultCallback - ResponseItem(stream=null, status=Pull complete, progressDetail=ResponseItem.ProgressDetail(current=null, total=null, start=null), progress=null, id=2538d1d7e815, from=null, time=null, errorDetail=null, error=null, aux=null)
...
[14:01:07] :     [Step 3/3] 14:01:07.228 [tc-okhttp-stream-276714561] INFO    [registry/private/oracle/database:18c_xe] - Pull complete. 2 layers, pulled in 46s (downloaded 637 MB at 13 MB/s)
[14:01:07] :     [Step 3/3] 14:01:07.228 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: registry/private/oracle/database:18c_xe
...
[14:01:07]i:     [Step 3/3] Docker event: {"status":"pull","id":"registry/private/oracle/database:18c_xe","Type":"image","Action":"pull","Actor":{"ID":"registry/private/oracle/database:18c_xe","Attributes":{"name":"registry/private/oracle/database"}},"scope":"local","time":1588075267,"timeNano":1588075267227817791}
...
[14:01:08] :     [Step 3/3]  :: Spring Boot ::        (v2.2.6.RELEASE)
[14:01:08] :     [Step 3/3] 
[14:01:08] :     [Step 3/3] 2020-04-28 14:01:08.502 ERROR 47 --- [           main] o.s.boot.SpringApplication               : Application run failed
[14:01:08] :     [Step 3/3] 
[14:01:08] :     [Step 3/3] java.lang.IllegalStateException: Mapped port can only be obtained after the container is started
[14:01:08] :     [Step 3/3]     at org.testcontainers.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:174) ~[testcontainers-1.14.1.jar:na]
[14:01:08] :     [Step 3/3]     at org.testcontainers.containers.ContainerState.getMappedPort(ContainerState.java:129) ~[testcontainers-1.14.1.jar:na]
[14:01:08] :     [Step 3/3]     at org.testcontainers.containers.OracleContainer.getOraclePort(OracleContainer.java:95) ~[oracle-xe-1.14.1.jar:na]
[14:01:08] :     [Step 3/3]     at org.testcontainers.containers.OracleContainer.getJdbcUrl(OracleContainer.java:64) ~[oracle-xe-1.14.1.jar:na]
[14:01:08] :     [Step 3/3]     at ch.package.OracleFlywayDatabaseTest$Initializer.initialize(OracleFlywayDatabaseTest.java:35) ~[test-classes/:na]
[14:01:08] :     [Step 3/3]     at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:626) ~[spring-boot-2.2.6.RELEASE.jar:2.2.6.RELEASE]
...
[14:01:08] :     [Step 3/3]     at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
...
[14:01:08] :     [Step 3/3]     at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) ~[surefire-booter-2.22.2.jar:2.22.2]
[14:01:08] :     [Step 3/3] 
...
[14:01:08] :     [Step 3/3] org.testcontainers.containers.ContainerLaunchException: Container startup failed
[14:01:08] :     [Step 3/3] Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=registry/private/oracle/database:18c_xe, imagePullPolicy=DefaultPullPolicy())
[14:01:08] :     [Step 3/3] Caused by: java.time.format.DateTimeParseException: Text '2020-03-04T15:17:25.025952651+01:00' could not be parsed at index 29
[14:01:08] :     [Step 3/3] 

I'm not sure about the last exception, it does not seem to bad, the problem seems to be the oracle container we start 'is not visible'. The date in that DateTimeParseException is the created date of our oracle-xe image in our registry.

I tried creating the container also with the withNetwork option on the builder:

@Testcontainers
public abstract class OracleFlywayDatabaseTest {

  @Container
  private static final OracleContainer oracle =
        new OracleContainer("registry/private/oracle/database:18c_xe")
                // .withNetwork(Network.builder().id("custom_network").build())
                .withUsername("TESTUSR")
                .withPassword("TESTPWD");

If I investigate this locally using docker network inspect custom_network the database container started by Testcontainers is not in that network.

What is the correct way to put a container in that network? Meaning the same network the builder image initially starts in? Is id really the id that docker assigns to the network when it is created? (I tried that but maybe I was doing something wrong).

Answer 1

Here is my solution for org.testcontainers:testcontainers:1.16.3 inside a TeamCity agent who in turn is a docker container itself.

var agentName = System.getenv("AGENT_NAME");
GenericContainer container = ...;
container.withNetwork(new ExistingNetwork("tagent-docker_default"))
    .withCreateContainerCmdModifier(cmd -> cmd.withHostName("myoracle-" + agentName));
// you don't even need to expose 1521 port
// set other container params. waitingFor, etc
container.start();

then use "jdbc:oracle:thin:@myoracle-" + agentName + ":1521/ORCLPDB1"
PS: ruyk is there, no need to turn it off
ExistingNetwork thankfully borrowed from wemu's answer up there

Answer 2

We have found a way to make this work. It's not what one would call beautiful... but here is what currently works:

Create the custom network with docker commands ("custom_nework" is the network name in this example):

docker network ls|grep custom_network > /dev/null || docker network create --driver bridge custom_network

Then determine the id of that network:

network_id=`docker network inspect custom_network --format "{{.ID}}"`

and set is and environment variable.

In the Testcontainers test you can now reference this network by: (in the local environment in the IDE when you just run once test, we have to distinguish if there is a custom network (CI server), or if there is none (IDE))

import org.apache.commons.lang3.StringUtils;
import org.junit.jupiter.api.BeforeAll;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.util.TestPropertyValues;
import org.springframework.context.ApplicationContextInitializer;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ContextConfiguration;
import org.testcontainers.containers.OracleContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

@Testcontainers
@ContextConfiguration(initializers = OracleFlywayDatabaseTest.Initializer.class)
@DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD)
public abstract class OracleFlywayDatabaseTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(OracleFlywayDatabaseTest.class);

    private static final String NETWORK_ID = "NETWORK_ID";

    @Container
    protected static final OracleContainer oracle;

    static {
        String networkId = System.getenv(NETWORK_ID);
        if (StringUtils.isBlank(networkId)) {
            oracle = new OracleContainer("diemobiliar/minimized-oraclexe-image:18.4.0-xe");
        } else {
            oracle = new NetworkOracleContainer("diemobiliar/minimized-oraclexe-image:18.4.0-xe", networkId);
        }
        oracle.withUsername("AOO_TESTS").withPassword("AOO_TESTS");
    }

    @BeforeAll
    public static void setupOracle() {
        LOGGER.info("ORACLE 18 JDBC URL: " + oracle.getJdbcUrl());
    }

    public static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {

        @Override
        public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
            TestPropertyValues.of("spring.datasource.platform=" + "ORACLE", //
                    "spring.datasource.url=" + oracle.getJdbcUrl(), //
                    "spring.datasource.username=" + oracle.getUsername(), //
                    "spring.datasource.password=" + oracle.getPassword()) //
                    .applyTo(configurableApplicationContext.getEnvironment());
            LOGGER.info("spring.datasource. Properties set.");
        }
    }

}

and the helping NetworkOracleContainer class:

import org.junit.runner.Description;
import org.junit.runners.model.Statement;
import org.testcontainers.containers.Network;

public class NetworkOracleContainer extends LocalOracleContainer {

    private static final String CONTAINER_NAME = "oracle";

    public NetworkOracleContainer(String dockerImageName, String networkId) {
        super(dockerImageName);
        this.withNetwork(new ExistingNetwork(networkId))
                .withCreateContainerCmdModifier(cmd -> cmd.withName(CONTAINER_NAME));
    }

    @Override
    public String getHost() {
        return CONTAINER_NAME;
    }

    @Override
    public Integer getOraclePort() {
        return 1521;
    }

    private static class ExistingNetwork implements Network {

        private final String networkId;

        ExistingNetwork(String networkId) {
            this.networkId = networkId;
        }

        @Override
        public String getId() {
            return networkId;
        }

        @Override
        public void close() {
            // noop
        }

        @Override
        public Statement apply(Statement base, Description description) {
            return base;
        }
    }

}

we haven't found a nicer way in the Testcontainers API to do this. Maybe in a newer version (currently at 1.14.3)