CODEWITHSUNDEEP
node.jsmongodbmongoose
pham cuong
pham cuong

Reputation: 861

InsertMany throw E11000 duplicate key error but insert successfully

I am using mongoose to insert bulk data to database. The insertMany throw E11000 duplicate key error although the data is saved succeed in database.

Here is my codes: 

  var data = [
      {_id: '598d7ae7c3148a6d853866e8'}, 
      {_id: '598d7afdc3148a6d853866f4'}
    ]
Movies.insertMany(data, function(error, docs) {});

This happen infrequently.

Appreciated any help.

Thanks

Upvotes: 4

Views: 8901

Answers (5)

Sergey Zhukov
Sergey Zhukov

Reputation: 534

Also consider checking option forceServerObjectId: true in insertMany params like this

const result = await usersCollection.insertMany(Object.values(usersData),{ ordered: false, forceServerObjectId: true });

Upvotes: 0

James111
James111

Reputation: 15903

I'm going to add this here for anyone who comes across this issue. I had the same exact issue and the reason I was getting this duplicate key error was because my old mongoose model used to enforce a UNIQUE constraint on one of my keys AND when I removed these unique constraints from my mongoose Model, I forgot to completely remove ALL existing indexes from the database. When I ran: db.mycollection.getIndexes(), I found that the old unique indexes were STILL in my database EVEN after removing all documents from each collection.

To get rid of these indexes I ran: db.mycollection.dropIndexes() in the shell.

Another tip I have for you: If you have a database seeding script that clears your collections & inserts fresh data (e.g for your test suite), I recommend you drop all indexes for each collection AFTER clearing your database. You can drop indexes for your models in mongoose like so: myMongooseModel.collection.dropIndexes()

Upvotes: 2

Danny Mora
Danny Mora

Reputation: 21

You should declare a function to find and update, and iterate the array data

set upset and new with true

  _storeLive(data) {
    shemaLive.findOneAndUpdate({
      _id: data._id
    }, data, {
      upsert: true,
      new: true
    }, (err, event) => {
      if (err) {} else {}
    });
  }


customArray.forEach(obj => _storeLive(obj));

Upvotes: 0

pham cuong
pham cuong

Reputation: 861

I had figured out the problem. It's a bug of Studio 3T. I drop all collections then import new collections then refresh collections tree. Some dropped collections are back.

Upvotes: 1

robertklep
robertklep

Reputation: 203231

MongoDB doesn't support transactions.

Because your operation is ordered (the default), MongoDB starts inserting documents until either all documents have been written, or it hits an error (for instance, when one of the _id's already exists in the database), at which point MongoDB will stop inserting the rest of the documents.

So if you try to insert 10 documents, and after 5 documents an error occurs, the first 5 documents will get inserted, but the rest won't. This is documented here.

That same document also suggests how to deal with this: disable ordering, because "With ordered [set] to false, the insert operation would continue with any remaining documents."

Movies.insertMany(data, { ordered : false }, function(error, docs) { ... });

Upvotes: 9

Related Questions