Load neo4j dump in test container during integration test run

Question

I am trying to write integration test for neo4j using spring boot. I am using test container. Can anyone help me how to load database dump file to testcontainer?

Answer 1

here's one way to do this (using current Neo4j 5.3). I have a dump file called neo4j.dump created from a local instance and I use withCopyFileToContainer to copy it to the container before it starts.

As I use the community edition in this example, there is no online backup/restore, but only dump/load. So therefor I have to change the startup command. The load needs to happen before Neo4j starts. I can't stop Neo4j in the container because it would stop the container.

Therefor I create a small shell script that executes the desired other command and than delegates to the original entry point.

This script is transferred with file mode 0100555, corresponding to r-xr-xr-x so that it is executable.

Finally, the container is started with the above script as command.

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.images.builder.Transferable;
import org.testcontainers.utility.MountableFile;

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class LoadDumpTest {

    Neo4jContainer<?> neo4j;

    Driver driver;

    @BeforeAll
    void initNeo4j() {

        neo4j = new Neo4jContainer<>("neo4j:5.3.0")
            .withCopyFileToContainer(MountableFile.forClasspathResource("neo4j.dump"),
                "/var/lib/neo4j/data/dumps/neo4j.dump")
            .withCopyToContainer(Transferable.of("""
                #!/bin/bash -eu
                /var/lib/neo4j/bin/neo4j-admin database load neo4j
                /startup/docker-entrypoint.sh neo4j
                """, 0100555), "/startup/load-dump-and-start.sh")
            .withCommand("/startup/load-dump-and-start.sh")
            .withLogConsumer(f -> System.out.print(f.getUtf8String()));
        neo4j.start();
        driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", neo4j.getAdminPassword()));
    }

    @Test
    void dataShouldHaveBeenLoaded() {
        try (var session = driver.session()) {
            var numNodes = session.run("MATCH (n) RETURN count(n)").single().get(0).asLong();
            Assertions.assertTrue(numNodes > 0);
        }
    }

    @AfterAll
    void stopNeo4j() {
        neo4j.stop();
    }
}

Edit: If you are on a recent enterprise edition, Christophe suggested the following solution on Twitter, which I do personally think it's superior, as it is less hacky.

https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup/blob/main/src/test/java/dev/ikwattro/Neo4j5RestoreBackupExampleTest.java

It makes use of seed URIs for databases. Copying or binding the resource in his example works the same as in mine.

Answer 2

If you're using Neo4j 5, I suggest you make backups instead of dumps. Using backups you can take advantage of the seedUri option when creating a database, meaning you can create a database from an URI pointing to a backup on disk ( or in the neo4j container ). https://neo4j.com/docs/operations-manual/current/clustering/databases/#cluster-seed-uri

Here is an example using Testcontainers

package dev.ikwattro;

import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

import static org.assertj.core.api.Assertions.assertThat;

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@Testcontainers(disabledWithoutDocker = true)
public class Neo4j5RestoreBackupExampleTest {

    @Container
    private Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5.3.0-enterprise")
            .withAdminPassword("password")
            .withEnv("NEO4J_dbms_memory_heap_max__size", "256M")
            .withEnv("NEO4J_dbms_databases_seed__from__uri__providers", "URLConnectionSeedProvider")
            .withClasspathResourceMapping("backups", "/backups", BindMode.READ_ONLY)
            .withEnv("NEO4J_ACCEPT_LICENSE_AGREEMENT", "yes");


    @BeforeAll
    void beforeAll() {
        neo4j.start();
        createDbFromBackup();
    }
    @Test
    void testCreatingDbFromBackup() {
        try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
            try (Session session = driver.session(SessionConfig.forDatabase("worldcup22"))) {
                var result = session.run("MATCH (n) RETURN count(n) AS c").single().get("c").asLong();
                assertThat(result).isPositive();
            }
        }
    }

    private void createDbFromBackup() {
        try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
            try (Session session = driver.session(SessionConfig.forDatabase("system"))) {
                session.run("""
                        CREATE DATABASE worldcup22 OPTIONS { existingData: "use", seedUri: "file:///backups/world-cup-2022-neo4j.backup"}
                        """);
            }
        }
    }

}

You can find a working maven project here https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup