CODEWITHSUNDEEP
mongodbpymongoaws-documentdb
theahura
theahura

Reputation: 433

Determine whether pymongo/documentdb has successfully inserted before moving on or trying another query

I am using pymongo on an AWS DocumentDB backend. I am trying to create a system where a user can post a query, and then use that query soon after posting. Something like:

client = pymongo.MongoClient()
client.db.queries.insert_one({'key': 'myKey', 'query': {'filter': 'test'}})
---
query = client.db.queries.find_one({'key': 'myKey'})
client.db.elements.find(query['query'])

This works most of the time, but sometimes the find_one on the third line will fail, returning no documents. I was having trouble understanding the documentation around how DocumentDB/Pymongo blocks for inserts, so I put together a little experiment:

import pymongo

client = pymongo.MongoClient()

print('prev: ', len(list(client.test.collection.find())))   # 0
print(client.test.collection.insert_one({'test': 'test'}))  # <pymongo.results.InsertOneResult object at 0x7f010806a0c0>
print('after: ', len(list(client.test.collection.find())))  # 0 -- I would expect this to be 1 if the insert succeeded

and I found that the insert was non-blocking.

Another example:

import pymongo

client = pymongo.MongoClient()

print('prev: ', len(list(client.test.collection.find())))   # 0
insert_result = client.test.collection.insert_one({'test': 'test'}))
print('inserted?', list(client.test.collection.find({'_id': insert_result.inserted_id})))  # [] -- I would expect this to have the inserted doc
print('after: ', len(list(client.test.collection.find())))  # 0 -- I would expect this to be 1 if the insert succeeded

Eventually, if I put a sleep or enough code between the insert and the finds, I will get results. So the insert is succeeding, just not immediately.

How am I supposed to determine whether a write has actually completed? I don't want to run another query unless I am sure the previous inserts have finished -- is there a way to make pymongo respect the insert order before running find queries?

Upvotes: 0

Views: 545

Answers (1)

Alex Blex
Alex Blex

Reputation: 37048

Change read preference to primary or primaryPreferred. Please get familiar with https://docs.aws.amazon.com/documentdb/latest/developerguide/how-it-works.html#durability-consistency-isolation to understand trade-offs. This part in particular:

Supported Read Preference Options

primary—Specifying a primary read preference helps ensure that all reads are routed to the cluster’s primary instance. If the primary instance is unavailable, the read operation fails. A primary read preference yields read-after-write consistency and is appropriate for use cases that prioritize read-after-write consistency over high availability and read scaling.

The following example specifies a primary read preference:

db.example.find().readPref('primary')

primaryPreferred—Specifying a primaryPreferred read preference routes reads to the primary instance under normal operation. If there is a primary failover, the client routes requests to a replica. A primaryPreferred read preference yields read-after-write consistency during normal operation, and eventually consistent reads during a failover event. A primaryPreferred read preference is appropriate for use cases that prioritize read-after-write consistency over read scaling, but still require high availability.

The following example specifies a primaryPreferred read preference:

db.example.find().readPref('primaryPreferred')

Upvotes: 1

Related Questions