Duplicate Index on mongoDB / Node.js

Question

Strange question / behaviour on my side: I have a mongoDB collection with a Index Unique on the _id Field.

In order to add an element to the collection and to be sure i'm not erasing the document if it already exists, I'm using this request:

db.getCollection('name').updateOne({
  "_id": "5bf9c75b478b493e9c989d96"
}, {
  "$setOnInsert": {
    "_id": "5bf9c75b478b493e9c989d96",
    "offset": 53,
    "data": {
      // some data
    }
  }
}, {
  'upsert' : true
})

If I'm not mistaken, this means: try to find the :id, if it doesn't exist, insert the beautiful structure. If the :id is found, the $setOnInsert won't be triggered and the request is ignored.

This works perfectly in robomongo, but when i'm trying to integrate it to my node.js program, it throws a duplicate Id exception if the structure with an existing :id is found, because it seems it still tries to write it.

Do we agree that the whole operation (finding, setting, modifying or not the collection) should be atomic ? I know that the problem is comming from the fact that I'm doing that in a micro services env, so many threads trying to execute the same operation on Mongo, that's why I NEED it to be atomic.

EDIT Following Alex's answer:

The mongo Error:

{
  "driver": true,
  "name": "MongoError",
  "index": 0,
  "code": 11000,
  "errmsg": "E11000 duplicate key error collection: eventSourcing.name index: _id_ dup key: { : ObjectId('5bfbd5a0478b493e9c989db0') }"
}

My code:

public saveIfAbsent(aggregate: T, writer: Writer<T, Schema>): Promise<UpdateWriteOpResult> {
  const element = writer(aggregate)

  if (element['_id'] !== aggregate._id) {
    throw new Error("should not happen")
  }

  return super.collection().then((collection) => {
    return collection.updateOne({
      '_id': aggregate._id
    }, {
      '$setOnInsert': element
    }, {
      'upsert' : true
    })
  }).catch((e) => {
    console.log(e)
    return Promise.reject(e)
  })
}

Any ideas on how I could make it work ? I want something able to write only if the structure is not here, and not doing anything otherwise.

Thanks

Edit:

It seems I'm trying to do something impossible ... or I'm doing it wrong.

const db = new Database()

const data = new Array<string>()
const length = 500; // user defined length

for (var i = 0; i < length; i++) {
  data.push('toto')
}

Promise.all(data.map((e) => db.db.then((db) => db.collection('name').updateOne({
  '_id': 1,
}, {
  '$setOnInsert': {
    'id': 1,
    'data': e
  }
}, {
  upsert: true
})))).then(() =>
  console.log('true'))

This piece of code tries to push in base { 'id': 'data': 'toto' }.

Expected behavior:

• First loop, it finds nothing then insert a { 'id': 'data': 'toto' } because the flag '$setOnInsert' is active

• All the other, { '_id': 1 } should return something, so '$setOnInsert' should never be written, so it should "pass" doing nothing

Current behavior:

• First loop, it finds nothing then insert a { 'id': 'data': 'toto' } because the flag '$setOnInsert' is active

• Crashes with a UnhandledPromiseRejectionWarning: MongoError: E11000 duplicate key error collection: eventSourcing.name index: _id_ dup key: { : 1 }. Somehow, it seems it has tried to write something within the '$setOnInsert' part => this seems to happen only if the collection name is dropped from the DB. if the collection is already existing, it doesn't seem to crash (surprisingly)

Native mongo DB driver for node JS v3.1.10

I'm trying to have the Expected behavior in my code, is it any way to do it ?

Answer 1

this.representationId(aggregate) does not return the same data/type that element._id has.

Either check it before:

if (this.representationId(aggregate) !== element._id) {
    throw new Error("Impossible happened!")
}

or use element._id in the query

return collection.updateOne({
    '_id': element._id
  }, {
    '$setOnInsert': element
  }, {
    'upsert': true
  })
})

or do both.