TestContainers and Error : "Failed to validate connection org.postgresql.jdbc.PgConnection" (raising a single container for all test classes)

Question

I have a problem when I try to run the tests one by one. The database connection is closed.

According to the documentation (Containers declared as static fields ...), I tried to make sure that my container was raised once for all tests.

I specifically used this to have the application context for Spring and a test-container raised once and used for all tests.

And it really is, because I do a check in every test:

      boolean running = getPostgreSQLContainer().isRunning();

        System.out.println(running);

That is, the tests are run automatically one by one.

• pom.xml

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.1</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

....

<properties>
        <java.version>11</java.version>
        <testcontainers.version>1.15.1</testcontainers.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <version.mapstruct>1.4.1.Final</version.mapstruct>
        <version.maven.compiler.plugin>3.8.1</version.maven.compiler.plugin>
        <version.embedded.postgresql.testcontainers>1.86</version.embedded.postgresql.testcontainers>
        <version.spring.cloud.starter>2.2.6.RELEASE</version.spring.cloud.starter>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>postgresql</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.mapstruct</groupId>
            <artifactId>mapstruct</artifactId>
            <version>${version.mapstruct}</version>
        </dependency>

        <dependency>
            <groupId>org.mapstruct</groupId>
            <artifactId>mapstruct-processor</artifactId>
            <version>${version.mapstruct}</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.liquibase</groupId>
            <artifactId>liquibase-core</artifactId>
        </dependency>

        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <scope>test</scope>
        </dependency>

    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.testcontainers</groupId>
                <artifactId>testcontainers-bom</artifactId>
                <version>${testcontainers.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

• TestPostgresContainer

@Testcontainers
@TestPropertySource("classpath:application.properties")
public class TestPostgresContainer {

    private static String dataBaseName;
    private static String userNameBase;
    private static String passwordBase;

    public TestPostgresContainer() {
    }

    private static DockerImageName postgres;

    static {
        postgres = DockerImageName.parse("postgres:13.1");
        dataBaseName = PropertiesExtractor.getProperty("database.name.test.container");
        userNameBase = PropertiesExtractor.getProperty("username.testcontainer");
        passwordBase = PropertiesExtractor.getProperty("password.testcontainer");
    }

    @SuppressWarnings("rawtypes")
    @Container
    private static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer(postgres)
            .withDatabaseName(dataBaseName)
            .withUsername(userNameBase)
            .withPassword(passwordBase)
            .withStartupTimeout(Duration.ofSeconds(600));


    @SuppressWarnings("rawtypes")
    public static PostgreSQLContainer getPostgreSQLContainer() {
        return postgreSQLContainer;
    }

    /**
     * It need for Spring boot 2.2.6 and higher.
     */
    @DynamicPropertySource
    static void properties(DynamicPropertyRegistry propertyRegistry){

        propertyRegistry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl);
        propertyRegistry.add("spring.datasource.username", postgreSQLContainer::getUsername);
        propertyRegistry.add("spring.datasource.password", postgreSQLContainer::getPassword);
    }

• TestcontainersSpringBootClassruleApplicationTests

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class TestcontainersSpringBootClassruleApplicationTests extends TestPostgresContainer {

    @Autowired
    protected TestRestTemplate testRestTemplate;


    @Test
    @DisplayName("Should start the container")
    public void test() {

        boolean running = getPostgreSQLContainer().isRunning();

        System.out.println(running);
    }
}

• EmployeeRestControllerTest

class EmployeeRestControllerTest extends TestcontainersSpringBootClassruleApplicationTests {

    private static EmployeeDto employeeDto;

    @BeforeAll
   static void createUser(){

        PostgreSQLContainer postgreSQLContainer = getPostgreSQLContainer();

        employeeDto = EmployeeDto
                .newBuilder()
                .firstName("Joanna")
                .lastName("Soyer")
                .country("germany")
                .build();
    }

    @Transactional
    @Test
    void addEmployee() {
        boolean running = getPostgreSQLContainer().isRunning();

        System.out.println(running);

        String url = "/employees/addEmployee";

        HttpEntity<EmployeeDto> entity = new HttpEntity<>(employeeDto);

        ResponseEntity<EmployeeDto> employeeDtoResponseEntity =
                testRestTemplate.exchange(url, HttpMethod.POST, entity, EmployeeDto.class);

        HttpStatus statusCode = employeeDtoResponseEntity.getStatusCode();
        assertThat(statusCode, is(HttpStatus.OK));
    }

    @Test
    void getAllEmployees() {
    }
}

Test classes are located in different directories

• application.properties

spring.main.banner-mode=off
spring.datasource.initialization-mode=always

## PostgreSQL for TestContainers
database.name.test.container=integration-tests-db
username.testcontainer=root
password.testcontainer=root

spring.datasource.hikari.max-life = 600000 

spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true

spring.liquibase.change-log=classpath:/db/changelog/db.changelog-master-test.xml

I use the liquibase.

Wnen is been running all test :

com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@698af960 (Соединение уже было закрыто). Possibly consider using a shorter maxLifetime value.

I set a value

spring.datasource.hikari.max-life = 600000

It doesn't help.

But when I run one test class at a time, then there is no error.

I found this:

Running container in daemon mode By default database container is being stopped as soon as last connection is closed. There are cases when you might need to start container and keep it running till you stop it explicitly or JVM is shutdown. To do this, add TC_DAEMON parameter to the URL as follows:

jdbc:tc:mysql:5.7.22:///databasename?TC_DAEMON=true

But, Where can I add TC_DAEMON=true to ?

I don't specify the url itself directly.. It is done TestContainers.

With this parameter database container will keep running even when there're no open connections.

Update

I edited this:

@Container
    private static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer(postgres)
            .withDatabaseName(dataBaseName)
            .withUsername(userNameBase)
            .withPassword(passwordBase);

• and

    @DynamicPropertySource
    static void registerPgProperties(DynamicPropertyRegistry propertyRegistry){

        String jdbcUrlPart = getPostgreSQLContainer().getJdbcUrl();
        String jdbcUrlFull = jdbcUrlPart + "&TC_DAEMON=true";

        propertyRegistry.add("integration-tests-db", getPostgreSQLContainer()::getDatabaseName);
        propertyRegistry.add("spring.datasource.username", getPostgreSQLContainer()::getUsername);
        propertyRegistry.add("spring.datasource.password", getPostgreSQLContainer()::getPassword);
        propertyRegistry.add("spring.datasource.url", () -> jdbcUrlFull);
    }

That didn't help either.

I've run out of ideas.

Does anyone have any idea how to fix this ?

Answer 1

You are using the JUnit Jupiter extension (@Testcontainers) that controls the lifecycle of your containers. This extension supports two modes:

1. containers that are restarted for every test method
2. containers that are shared between all methods of a test class

So for your setup Testcontaienrs starts a database for each test class that is not shared between several test classes but test methods within the same test class.

With all the code hierarchy and code updates in your question, it's hard to say where the issue is coming from.

If you want to re-use a database container and only start it once, take a look at Singleton containers and the reusability feature.

PS: You don't need @RunWith(SpringRunner.class) as you are running your tests with JUnit 5 and the JUnit Jupiter extension (SpringExtension) is already registered as part of @SpringBootTest

Answer 2

Solution

public class TestPostgresContainer {

    @SuppressWarnings("rawtypes")
    private static PostgreSQLContainer postgreSQLContainer;

    public TestPostgresContainer() {
    }


    static {
        DockerImageName postgres = DockerImageName.parse("postgres:13.1");

        postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer(postgres)
                .withReuse(true);

        postgreSQLContainer.start();
    }
....

   @SuppressWarnings("unused")
    @DynamicPropertySource
    static void registerPgProperties(DynamicPropertyRegistry propertyRegistry) {

        String jdbcUrl = getPostgreSQLContainer().getJdbcUrl();

        propertyRegistry.add("integration-tests-db", getPostgreSQLContainer()::getDatabaseName);
        propertyRegistry.add("spring.datasource.username", getPostgreSQLContainer()::getUsername);
        propertyRegistry.add("spring.datasource.password", getPostgreSQLContainer()::getPassword);
        propertyRegistry.add("spring.datasource.url", () -> jdbcUrl);
    }

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class TestcontainersSpringBootClassruleApplicationTests extends TestPostgresContainer {

    @Autowired
    protected TestRestTemplate testRestTemplate;

    @Test
    @DisplayName("Should start the container")
    public void test() {

        boolean running = getPostgreSQLContainer().isRunning();

        System.out.println("Is The container run : " + running);
    }

}

class EmployeeRestControllerTest extends TestcontainersSpringBootClassruleApplicationTests {

    private static EmployeeDto employeeDto;

   @BeforeAll
   static void createUser(){
...

public class EmployeeReadRestControllerTest extends TestcontainersSpringBootClassruleApplicationTests {

    @Test
    void getAllEmployees() {

1. I didn't use the annotation @Testcontainers

2. I didn't use resources/testcontainers.properties (I don't quite understand why this file is needed. After all, everything works as it is.)

3. It's important:

## PostgreSQL for TestContainers
database.name.test.container=integration-tests-db
username.testcontainer=root
password.testcontainer=root

• and

 @DynamicPropertySource
    static void registerPgProperties(DynamicPropertyRegistry propertyRegistry) {

        String jdbcUrl = getPostgreSQLContainer().getJdbcUrl();

        propertyRegistry.add("integration-tests-db", getPostgreSQLContainer()::getDatabaseName);
        propertyRegistry.add("spring.datasource.username", getPostgreSQLContainer()::getUsername);
        propertyRegistry.add("spring.datasource.password", getPostgreSQLContainer()::getPassword);
        propertyRegistry.add("spring.datasource.url", () -> jdbcUrl);
    }

4. This is not necessary (at least it is not necessary to run Testcontainers). I don't know why.

@TestPropertySource("classpath:application.properties")

Instead @DynamicPropertySource works.

I wand to add few comments:

• 1 If you use the method .withReuse(true), after When You have fifnished to run test. You must clean a docker space from containers .

in this case, Ryuk won't do it for you.

 postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer(postgres)
      .withDatabaseName("test")
      .withUsername("name")
      .withPassword("password")
      .withReuse(true);

• 2. If you only want to increase time run your tests ,Then I recommend to use Singleton containers. In that case, Ryuk will do it for you.

Thanks.