CODEWITHSUNDEEP
pythonmongodbamazon-web-servicespymongoaws-documentdb
ruhufu
ruhufu

Reputation: 81

Pymongo connection to DocumentDB cluster

I'm facing some problems in order to programatically connect python to our AWS DocumentDB (AWS version of MongoDB) instance in a local environment. We tried the following steps:

  1. To start our experiment we used the recommended AWS tutorial to create the python connection: https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html.

  2. Since we want to initially set this locally, we do an SSH connection to our cluster using the following command: ssh -i "example.pem" -L 27019:docdb.cluster-XXXXXXX.region.docdb.amazonaws.com:27017 [email protected] -N -v This command works fine and we are able to create the tunnel and connect local port 27019 to our cluster. One important note regarding this, is that in order to reach the cluster we have to activate a VPN network.

  3. We adapted our code to connect to our local port:

    from pymongo import MongoClient,ReadPreference
    import urllib
    import ssl
    
    username=urllib.parse.quote_plus("username")
    password=urllib.parse.quote_plus("password")
    port=27019
    host="localhost"
    dbName="general"
    
    
    dbUri = f'mongodb://{username}:{password}@{host}:{port}/?tls=true&tlsCAFile=./rds-combined-ca-bundle.pem&retryWrites=false'
    print(dbUri)
    
    client = MongoClient(dbUri)
    print(client.list_database_names())

Running the above example we get as error:

pymongo.errors.ServerSelectionTimeoutError: hostname '127.0.0.1' doesn't match either of 'docdb.XXXXXXX.region.docdb.amazonaws.com', 'docdb.cluster-XXXXXXX.region.docdb.amazonaws.com', 'docdb.cluster-ro-XXXXXXX.region.docdb.amazonaws.com', Timeout: 30s, Topology Description: <TopologyDescription id: 61e7f8c811a815088e97e5a7, topology_type: Unknown, servers: [<ServerDescription ('127.0.0.1', 27019) server_type: Unknown, rtt: None, error=CertificateError("hostname '127.0.0.1' doesn't match either of 'docdb.XXXXXXX.region.docdb.amazonaws.com', 'docdb.cluster-XXXXXXX.region.docdb.amazonaws.com', 'docdb.cluster-ro-XXXXXXX.region.docdb.amazonaws.com'",)>]>
2022-01-19 11:41:27,120| ERROR   | Could not establish connection from local ('127.0.0.1', 27019) to remote ('ai-docdb-dev.cluster-cq8mixmxuhvt.eu-west-1.docdb.amazonaws.com', 27017) side of the tunnel: open new channel error:

This indicates 2 things: First our code is correctly using the .pem certificate and is able to get our cluster names. Second, the code is not able to understand that localhost is an address and inspect the local port 27019 for reaching the cluster.

To try to overcome this issue we also tried to substitute our code variable host to 127.0.01, however the result was the same. We also tried to tweak several parameter of the pymongo function MongoClient https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient , but the result remained the same.

Thanks

Upvotes: 2

Views: 8025

Answers (4)

George Semenov
George Semenov

Reputation: 11

this connection string worked for me:

mongodb://mongodbuser:{username}:{password}@{host}:27017/?tls=true&tlsCAFile=./rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false&tlsAllowInvalidHostnames=true&directConnection=true

Also directConnection - for aws it must be false, for local true (when use tunnel for example)

Upvotes: 1

tmcallaghan
tmcallaghan

Reputation: 1302

Add &tlsAllowInvalidHostnames=true to the URI to skip the hostname check.

Upvotes: 1

ruhufu
ruhufu

Reputation: 81

After a lot of non-successful attempts we finally managed to find a solution!

With pymongo 3.X.X, we were able to connect to our documentDb instance by adding the parameter tlsInsecure=true to the original URI, which is passed to our MongoClientfunction.

For pymongo 4.X.X we had to add an extra parameter directConnection=true to make things work.

The host variable can be set to localhost or 127.0.0.1 and username and password can be encripted with urlib or not.

Upvotes: 6

Belly Buster
Belly Buster

Reputation: 8834

It looks like the certificate checking is validating the hostname you are using, so we can trick it by using the actual hostnames and mapping them in the hosts file to 127.0.0.1.

Add each of the docdb.XXXXXXX.region.docdb.amazonaws.com to your calling system's hosts file, e.g.

127.0.0.1 docdb.XXXXXXX-1.region.docdb.amazonaws.com
127.0.0.1 docdb.XXXXXXX-2.region.docdb.amazonaws.com
127.0.0.1 docdb.XXXXXXX-3.region.docdb.amazonaws.com

Then change the host variable:

host=f"docdb.XXXXXXX-1.region.docdb.amazonaws.com:{port},docdb.XXXXXXX-2.region.docdb.amazonaws.com:{port},docdb.XXXXXXX-3.region.docdb.amazonaws.com:{port}"

And then change the URI to remove the port as it's now in the host variable:

dbUri = f'mongodb://{username}:{password}@{host}/?tls=true&tlsCAFile=./rds-combined-ca-bundle.pem&retryWrites=false'

Upvotes: 2

Related Questions