CODEWITHSUNDEEP
javascriptnode.jsmongodbnpmbulkinsert
Volox
Volox

Reputation: 1108

Avoid mongodb bulk insert duplicate key error

How can I execute a bulk insert and continue in case of duplicate key error?

I have a collection with an unique index on the id field (not _id) and some data in it. Then I get more data and I want to add only the non-present documents to the collection.

I have the following code:

let opts = {
  continueOnError: true, // Neither
  ContinueOnError: true, // of
  keepGoing: true,       // this
  KeepGoing: true,       // works
};
let bulk = collection.initializeUnorderedBulkOp( opts );
bulk.insert( d1 );
bulk.insert( d2 );
bulk.insert( d3 );
...
bulk.insert( dN );
let result = yield bulk.execute( opts ); // this keep throwing duplicate key error

And I just want to ignore the errors and let the bulk finish with all the queued operations.

I searched in npm module api and in the MongoDB api for Bulk, initializeUnorderedBulkOp and the docs for Bulk write with no luck.

Also in the docs for Unordered Operations they say:

Error Handling

If an error occurs during the processing of one of the write operations, MongoDB will continue to process remaining write operations in the list.

Which is not true (at least in my case)

Upvotes: 18

Views: 20723

Answers (2)

user6184932
user6184932

Reputation:

Ordered Insert in MongoDB

db.hobbies.insertMany([{_id: "yoga", name: "Yoga"}, {_id: "cooking", name: "Cooking"}, {_id: "hiking", name: "Hiking"}], {ordered: true})

{ordered: true} is the default behaviour of insert statements

Unordered Insert in MongoDB

If you want mongodb to continue trying to insert other documents even after one or more failing due to any reason, you must set ordered to false. See example below:

db.hobbies.insertMany([{_id: "yoga", name: "Yoga"}, {_id: "cooking", name: "Cooking"}, {_id: "hiking", name: "Hiking"}], {ordered: false})

Upvotes: 9

Stanislav Prusac
Stanislav Prusac

Reputation: 778

You can use db.collection.insertMany(), (new in version 3.2.) with:

ordered:false

With ordered to false, and in case of duplicate key error, the insert operation would continue with any remaining documents.

Here is link to documentation: https://docs.mongodb.com/v3.2/reference/method/db.collection.insertMany/

Upvotes: 11

Related Questions