tech-ebe
tech-ebe

Reputation: 91

How to seed a MongoDB single node replica set when the container starts

I was using a standalone MongoDB instance and had the following:

Docker Compose:

  database:
    build:
      context: .
      dockerfile: ./db/Dockerfile.mongodb
    ports:
      - 12000:27017
    volumes:
      - ./data/data:/data/db
    environment:
      - MONGO_INITDB_DATABASE=DB

./db/Dockerfile.mongodb:

FROM mongo:latest

COPY ./db/seed /import
COPY ./db/import/import.sh /docker-entrypoint-initdb.d/
RUN chmod 777 /docker-entrypoint-initdb.d/import.sh

./db/import/import.sh:

#!/bin/bash
for f in /import/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --db=DB --collection=$name --jsonArray --file=$f
done

This would seed the database with .json files from ./db/seed/ directory whenever the container was started. Once it was up, my database would be in a ready to use state (for development, running automated tests, etc)

The problem with that setup is that I cannot use transactions in MongoDB.

Afterwards, I setup a single node replica set (to enable transactions) using the following:

Docker Compose:

services:
  mongo:
    build:
      context: .
      dockerfile: ./db/Dockerfile.mongodb
    command: [--replSet, rs0, --bind_ip_all, --port, "12000"]
    ports:
      - 12000:12000
    environment:
      - MONGO_INITDB_DATABASE=DB
    healthcheck:
      test: test $$(mongosh --port 12000 --quiet --eval "try {rs.initiate({_id:'rs0',members:[{_id:0,host:\"mongo:12000\"}]})} catch(e) {rs.status().ok}") -eq 1
      interval: 10s
      start_period: 0s

./db/Dockerfile.mongodb is same as above

./db/import/import.sh:

#!/bin/bash

for f in /import/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --host=rs0/0.0.0.0:12000 --db=DB --collection=$name --jsonArray --file=$f
done

It took a lot of experimentation with the docker compose to even get the single node replica set working, because a lot of examples on the internet do not work. The ones that do work seem to all use the healthcheck mechanism to initialize the replica set since this gets run after container has been started. This causes a race condition with the import.sh script because the database is not setup as a replica set when the data load tries to run and it causes errors and the container stops.

How can I seed the replica set with data from .json files, without manually running mongoimport (or similar) commands after the container is up. I would like to be able to run the docker compose up and have the replica set db seeded and ready to go. I cannot find any examples of how this is done.

----------------------------UPDATE---------------------------------

Based on answer from @diego-freniche, I came up with a solution. I do not need a custom Dockerfile anymore (since that was used to just import the data in a non replica set standalone instance).

Docker Compose:

services:
  mongo:
    image: mongo:latest
    command: [--replSet, rs0, --bind_ip_all, --port, "12000"]
    ports:
      - 12000:12000
    environment:
      - MONGO_INITDB_DATABASE=DB
    healthcheck:
      test: test $$(mongosh --port 12000 --quiet --eval "try {rs.initiate({_id:'rs0',members:[{_id:0,host:\"mongo:12000\"}]})} catch(e) {rs.status().ok}") -eq 1
      interval: 10s
      start_period: 0s
  mongo_seeder:
    image: mongo:latest
    depends_on:
      database:
        condition: service_healthy
    volumes:
      - ./tools/db:/tmp/db/
    environment:
      - MONGO_INITDB_DATABASE=DB
      - MONGOIMPORT_HOST_STRING=rs0/database:12000
    command: bash /tmp/db/import/import.sh

./tools/db/import/import.sh

#!/bin/bash

for f in /tmp/db/seed/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --host=$MONGOIMPORT_HOST_STRING --db=$MONGO_INITDB_DATABASE --collection=$name --jsonArray --file=$f
done

Upvotes: 0

Views: 95

Answers (1)

Diego Freniche
Diego Freniche

Reputation: 5414

This is the docker-compose.yml I use it to start MongoDB Community Edition and run a script when the server is up and running.

The key is that service mongodb_create_users depends on mongodb. So until mongodb is not fully loaded and the db running (I check that using healthcheck) mongodb_create_users does not start.

In this case I run a mongosh script, but you can run any other import script you need. mongosh accepts a JS file with commands, the file create-mongodb-user.js is in my current directory (where this compose file is)

services:
  # a MongoDB instance
  mongodb:
    image: mongodb/mongodb-community-server
    ports:
      - "27017:27017"
    environment:
      - MONGO_INITDB_ROOT_USERNAME=mongodb
      - MONGO_INITDB_ROOT_PASSWORD=password
    volumes:
      - type: bind
        source: ./data
        target: /data/db
      - ./:/tmp/import
    healthcheck:
      test: mongosh --eval 'db.hello()'
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 2s
    # mongodb://dvds:password@localhost:27017/dvds
  mongodb_create_users:
    image: mongodb/mongodb-community-server
    depends_on:
      mongodb:
        condition: service_healthy
    volumes:
      - ./:/tmp/import
    command: mongosh -u mongodb -p password --host mongodb admin -f /tmp/import/create-mongodb-user.js

Upvotes: 0

Related Questions