CODEWITHSUNDEEP
node.jsmongodbperformance
Adrian
Adrian

Reputation: 153

Slower mongodb queries for big documents

I have a mongodb database with just 2 documents. Both have the same structure:

{ "general" { "name": "abc", "sid": "435435"},"resources":[{"id":1,"cnt":20}] "messages" : []}

The small document has 0 objects in messages, the big 1000. I counted the signs in both documents: small: 28000 big: 450000

I am using nodeJS with the regular mongodb driver to access the documents and I have an index set to "general.sid".

Now I'm requesting the documents by their general.sid. And the times differ for both documents a lot! I receive the document, do some calculation and update the documents general.resources.

I print the time before and after receiving and updating the document I did this queries several times:

Receiving: Small document timespan: 1-2ms

Receiving: Big document timespan: 7-20ms

Writing: Small document timespan: 1-2ms

Writing: Big document timespan: 5-10ms

code for the receiving function:

db.get().collection('player_data').find({"general.sid":UID}).limit(1).toArray(function (err, result){
  if(err){
    reject(null);
  }
  resolve(result);
});

Why are there such big differences?

I am not a pro at mongodb or nodejs so please tell me if you need any other data!

Upvotes: 1

Views: 59

Answers (1)

Tom Slabbaert
Tom Slabbaert

Reputation: 22316

The difference comes from what you already said, the document size difference.

An indexed query first scan the index B-tree and once a match is found it starts fetching the documents, while the index scan time for a specific document can vary ( depending on the tree size, and where in the tree that document is located ). In your case, a collection with 2 documents and a simple index on a none array field the index scan portion of the query will be identical.

This leads us to the next part of the find query, reading the matched document into memory, and here there is no surprise, more bytes to load into memory. more time the query will take. It's harder to give an exact explanation for the write function as not many people know exactly how Mongo update a document but I imagine if update a field that's not the messages array the difference will also be negligible.

If you're asking for a better "design pattern" for your db it's entirely dependant on your day to day needs, but in order to fix this current issue I would suggest saving the messages in a separate collection.

Upvotes: 1

Related Questions