CODEWITHSUNDEEP
pythonmongodbpymongo
stav
stav

Reputation: 1755

Errors when iterating mongodb cursor (not because of timeouts)

Running a simple find on a large collection (250K items) in pymongo and iterating over the cursor. The cursor iteration fails, every time in a different place. All the documentation on the web on this talk about timeouts being the issue. However, for me this issue persists even though I've disabled the timeout, and am using a modest batch_size of 5.

I don't think this is pymongo issue since this happens when using other drivers and clients as well.

This is the pymongo code:

cursor = collection.find({}, no_cursor_timeout=True).max_time_ms(99999999).limit(50000).batch_size(5)

This will error at some point either with 

pymongo.errors.OperationFailure was unhandled by user code
Message: collection dropped between getMore calls

or

pymongo.errors.CursorNotFound was unhandled by user code
Message: Cursor not found, cursor id: 3771706747850

What could be the issue?

Upvotes: 0

Views: 1202

Answers (1)

A. Jesse Jiryu Davis
A. Jesse Jiryu Davis

Reputation: 24007

Another program connected to your database is periodically dropping the collection. I suggest you check your MongoDB server logs to confirm that the collection is dropped. You'll see an entry like:

2017-01-18T20:40:03.783-0500 I COMMAND  [conn83] CMD: drop test.collection

Upvotes: 3

Related Questions