CODEWITHSUNDEEP
pythonmongodbpymongodatabase
Vvan_G
Vvan_G

Reputation: 215

How to handle DuplicateKeyError in MongoDB (pyMongo)?

Could anybody please tell me how to handle DuplicateKeyError in MongoDB?

I am writing a python script, where I move several docs from two different collections into a third one. There is a small overlap between the two collections due to having a few identical documents (with identical ObjectId). This results in the following:

DuplicateKeyError: E11000 duplicate key error collection: admin.collection_test index: id dup key: { : ObjectId('593a920b529e170d4b8fbf72') }

In order to get rid of the error I use: 

try:
    do something
except pymongo.errors.DuplicateKeyError:
    pass

I expect by using of the "try-except" to move all non-crossing documents to the third collection, but instead the script just peacefully stops running once a first overlap (an already existing document in the collection) appears. Would appreciate any help a lot!

Upvotes: 17

Views: 27447

Answers (3)

Rajiv Sharma
Rajiv Sharma

Reputation: 7122

CONNECT TO DATABASE

connection = pymongo.MongoClient("192.168.2.202", 27017)

CREATE DATABASE

database = connection['my_database']

CREATE COLLECTION

collection = database['my_collection']

INSERT DOCUMENTS IN COLLECTION

url="http://some/api/url/path/?format=json"
data = {
    '_id': url,
    'timestamp': datetime.datetime.now(),
    'data': {
        'XX': 1,
        'YY': 2,
        'ZZ':  3
    }
}

TO AVOID DUPLICATES - THIS WILL CREATE NEW DOCUMENT IF SAME ID NOT EXIST

collection.update_one({'_id': url},  {"$set": data}, upsert=True)

Upvotes: -2

daemon24
daemon24

Reputation: 1438

for doc in documents:
    client.update_one({'_id': doc['_id']}, doc, upsert=True)

You can use update_one with upsert=True. This updates doc with new doc if doc exists already otherwise it creates new doc.

Upvotes: 15

Johnny Metz
Johnny Metz

Reputation: 5975

If you're iterating over the documents, try using continue instead of pass.

for doc in documents:
    try:
        # insert into new collection
    except pymongo.errors.DuplicateKeyError:
        # skip document because it already exists in new collection
        continue

Upvotes: 30

Related Questions