CODEWITHSUNDEEP
mongodbamazon-web-serviceskubernetes
Confounder
Confounder

Reputation: 555

Connecting from external sources to MongoDB replica set in Kubernetes fails with getaddrinfo ENOTFOUND error but standalone works

I have a MongoDB replica set running in Kubernetes (AWS EKS), created using Helm charts from Bitnami. The services are configured to be external facing and set to NodePort.

mongo-mongodb-0-external         NodePort    10.100.83.252  27017:30030/TCP

mongo-mongodb-1-external         NodePort    10.100.15.184  27017:30031/TCP

mongo-mongodb-2-external         NodePort    10.100.90.128  27017:30032/TCP

mongo-mongodb-arbiter-headless   ClusterIP   None           27017/TCP

mongo-mongodb-headless           ClusterIP   None           27017/TCP

On my laptop, I can connect Mongo CLI to the replica but connection fails from MongoDB Compass and Studio 3T.

The following works from Mongo CLI from my laptop...

mongo 'mongodb://root:mypassword@k8s_node_ip:30030,k8s_node_ip:30031,k8s_node_ip:30032/mydb?authSource=admin'

...and this works...

mongo admin --host "k8s_node_ip:30030,k8s_node_ip:30031,k8s_node_ip:30032" --authenticationDatabase admin -u root -p mypassword

...but this fails in MongoDB Compass and Studio 3T...

mongodb://root:mypassword@k8s_node_ip:30030,k8s_node_ip:30031,k8s_node_ip:30032/mydb?authSource=admin

The error message is:

getaddrinfo ENOTFOUND mongo-mongodb-0.mongo-mongodb-headless.mynamespace.svc.cluster.local

Bizarrely, the following standalone connection works in Studio 3T:

mongodb://root:mypassword@k8s_node_ip:30030/?serverSelectionTimeoutMS=5000&connectTimeoutMS=10000&authSource=admin&authMechanism=SCRAM-SHA-256

Upvotes: 2

Views: 3949

Answers (2)

Mohammad Faisal Khan
Mohammad Faisal Khan

Reputation: 31

Make sure that you have added replica set nodes in the host machine in etc/hosts file. Just like the below example -

127.0.0.1   mongoset1 mongoset2 mongoset3

Note - 127.0.0.1 is your host machine and mongoset1, mongoset2 and mongoset3 are the nodes (members) of the replicaset.

Upvotes: 3

Joe
Joe

Reputation: 28336

When connecting to a replica set, the host:port pairs in the connection string are a seedlist.

The driver/client will attempt to connect to each host in the seedlist in turn until it gets a connection.

It runs the isMaster command to determine which node is primary, and to get a list of all replica set members.

Then is drops the original seedlist connection, and attempts to connect to each replica set member using the host and port information retrieved.

The host information returned by the isMaster usually matches the entry in rs.conf(), which are the hostnames used to initiate the replica set.

In your Kubernetes cluster, the nodes have internal hostnames that are used to initiate the replica set, but that your external clients can't resolve.

In order to get this to work, you will need to have the mongod nodes isMaster command return a different set of hostnames depending on where the client request is coming from. This is similar to split-horizon DNS.

Look over the Deploy a Replica Set documentation for mongodb/kubernetes, and the replicaSetHorizons setting.

Upvotes: 5

Related Questions