Connect to a remote dockerized MongoDB replica set

Question

In my remote machine, I've set up a docker container machine that I manage using docker-compose. I created 3 docker containers for each MongoDB instance I want in my replica set

mongodb_01:
    image: mvertes/alpine-mongo
    entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
    restart: always
    ports:
      - 10001:27017
    volumes:
      - ./mongodb/01:/data/db

mongodb_02:
    image: mvertes/alpine-mongo
    entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
    restart: always
    depends_on:
       - mongodb_01
    ports:
      - 10002:27017
    volumes:
      - ./mongodb/02:/data/db

mongodb_03:
    image: mvertes/alpine-mongo
    entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
    restart: always
    depends_on:
      - mongodb_02
    ports:
      - 10003:27017
    volumes:
      - ./mongodb/03:/data/db

I also configured the replica set. and this is an excerpt:

"_id" : "rs0",
...
"members" : [
    {
        "_id" : 0,
        "host" : "mongodb_01:27017",
        ...
    },
    {
        "_id" : 1,
        "host" : "mongodb_02:27017",
        ...
    },
    {
        "_id" : 2,
        "host" : "mongodb_03:27017",
        ...
    }
],
...
}

Everything works fine, and intra-communications between other docker images and this replica set works fine using the connection string

mongodb://mongodb_01:27017,mongodb_02:27017,mongodb_03:27017/<database>?replicaSet=rs0

The problem is when I need to connect a remote client to this replica set. For example, using mongoose via node on my dev machine I get:

MongoNetworkError: failed to connect to server [mongodb_02:27017] on first connect [MongoNetworkError: getaddrinfo ENOTFOUND mongodb_02 mongodb_02:27017]

Sometimes it fails on mongodb_03.

Edit: as pointed out, here's my connection string from remote machine:

mongodb://<remote-host>:10001,<remote-host>:10002,<remote-host>:10003/<database>?replicaSet=rs0

Edit 2: using a client like Mongodb Compass I can successfully connect to the single instances correctly. When I add the replicaset, i got the error. So I tried to create a dummy container with mongodb (using mongo:latest).

$ docker run -it mongo:latest bash

and running

mongo mongodb://<remote-host>:10001,<remote-host>:10002,<remote-host>:10003/<database>?replicaSet=rs0

I get

MongoDB shell version v4.0.6
connecting to: mongodb://<remote-host>:10001,<remote-host>:10002,<remote-host>:10003/<database>?gssapiServiceName=mongodb&replicaSet=rs0
2019-03-04T16:12:54.375+0000 I NETWORK  [js] Starting new replica set monitor for rs0/<remote-host>:10001,<remote-host>:10002,<remote-host>:10003
2019-03-04T16:12:54.377+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to <remote-host>:10003 (1 connections now open to <remote-host>:10003 with a 5 second timeout)
2019-03-04T16:12:54.377+0000 I NETWORK  [js] Successfully connected to <remote-host>:10001 (1 connections now open to <remote-host>:10001 with a 5 second timeout)
2019-03-04T16:12:54.378+0000 I NETWORK  [js] changing hosts to rs0/mongodb_01:27017,mongodb_02:27017,mongodb_03:27017 from rs0/<remote-host>:10001,<remote-host>:10002,<remote-host>:10003
2019-03-04T16:12:54.882+0000 W NETWORK  [js] Unable to reach primary for set rs0
2019-03-04T16:12:54.882+0000 I NETWORK  [js] Cannot reach any nodes for set rs0. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.

and so on.

Thanks for any help and suggestion !

Answer 1

I faced exactly same problem with you, and I've figured out it.

It is because that your remote client does not know 'mongo1:27017' host. It's is just used inside docker network only.

It is a bit tricky to explain how I've solved this problem. I'll show my docker-compose.yml first.

version: "3.3"
services:
  mongo-primary:
    container_name: mongo-primary
    hostname: mongo-primary
    image: mongo:4.0.11
    volumes:
      - $HOME/.dockerMongoRepl/primary/data/db:/data/db
      - $HOME/.dockerMongoRepl/keyfile:/data/keyfile
    extra_hosts:
      - "address.whichCanAccess.yourServer:192.168.1.xx"
    networks:
      - mongo-cluster
    ports:
      - 27017:27017
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: changeme
    command: --bind_ip_all --auth --keyFile /data/keyfile/mongo-cluster-key --replSet rs0 --enableMajorityReadConcern false
  mongo-secondary:
    container_name: mongo-secondary
    hostname: mongo-secondary
    image: mongo:4.0.11
    volumes:
      - $HOME/.dockerMongoRepl/secondary/data/db:/data/db
      - $HOME/.dockerMongoRepl/keyfile:/data/keyfile
    depends_on:
      - mongo-primary
    extra_hosts:
      - ""address.whichCanAccess.yourServer:192.168.1.xx""
    networks:
      - mongo-cluster
    ports:
      - 27018:27017
    restart: always
    command: --bind_ip_all -auth --keyFile /data/keyfile/mongo-cluster-key --replSet rs0 --enableMajorityReadConcern false
  mongo-arbiter:
    container_name: mongo-arbiter
    hostname: mongo-arbiter
    image: mongo:4.0.11
    volumes:
      - $HOME/.dockerMongoRepl/arbiter/data/arb:/data/arb
      - $HOME/.dockerMongoRepl/keyfile:/data/keyfile
    depends_on:
      - mongo-primary
    extra_hosts:
      - ""address.whichCanAccess.yourServer:192.168.1.xx""
    networks:
      - mongo-cluster
    ports:
      - 27019:27017
    restart: always
    command: --bind_ip_all --auth --keyFile /data/keyfile/mongo-cluster-key --replSet rs0 --enableMajorityReadConcern false

networks:
  mongo-cluster:

!Important part is 'extra_hosts'!! It can make containers access to host computer.

"address.WhichCanAccess.yourServer" <- in my case, my asus router has been set with asus ddns, so it will be XXX.asuscomm.com

"192.168.1.xx" <- IP addres which asus router has assigned the host computer

Maybe some configuration of those compose file are not required.

Run 3 containers with docker-compose.

Next, enter the primary's mongo shell, set replica like below

config = {
  "_id": "rs0",
  "members": [{
    "_id": 0,
    "host": "address.whichCanAccess.yourServer:27017"
  }, {
    "_id": 1,
    "host": "address.whichCanAccess.yourServer:27018"
  }, {
    "_id": 2,
    "host": "address.whichCanAccess.yourServer:27019",
    arbiterOnly: true
  }]
}

rs.initiate(config)

In this way, mongo containers will communicate each other through docker's host network, and it can be accessed from remote IP.

Answer 2

You can use localhost for this purpose. The compose will roughly look like:

version: "3"
services:
  mongodb:
    image: mongo:4.0.11
    ports:
      - "27017:27017"
    extra_hosts:
      - "localhost:0.0.0.0"
    volumes:
      - "./lambda/docker/mongod.conf:/etc/mongod.conf"

Then when initializing replicaset, make sure to set your host to localhost. Example:

{
    // ........
    "members" : [ 
        {
            "_id" : 0,
            "host" : "localhost:27017"
            // .........
        }
    ]
    // ...........
}

Tested on OSX. On linux/windows it may potentially have different behavior.

Answer 3

I ended up using a free version of Atlas for test and integration. Once the my work is complete, I'll use my own replica set on my servers.

That's a bit of a hassle, and I need to triple check everything before production without the support of dev tools and debuggers.