CODEWITHSUNDEEP
node.jsmongodb
Ruben Homs
Ruben Homs

Reputation: 591

Node.js & MongoDB performance issues

I'm having some weird performance issues with MongoDB. First I tried to get about ~1000 documents through the mongojs driver which did work, but finished the query in about 60+ seconds. Note that the documents are not big in any way!

Then I tried to do the same query but ran an explain() against the mongo shell to check if it really was just the query being so slow. Upon inspection it turned out that the query took only 4ms to complete! I have no idea what's causing this. Any ideas?

Some more details:

EDIT:

I ran wireshark and noticed the following:

  1. The query gets send to the server

  2. The server responds with 101 documents in under 1ms (always the same number)

  3. The client does a "Get More" request

  4. It takes about a minute or two before the server responds with the rest of the documents

  5. The "Get More" request and server responses with documents continues until it has sent everything. Which might take... forever for even the smallest set of around 1000 documents!

Here's a piece of code that I run in the node CLI:

/* Get all the documents (about 1000) */
var db = require('mongojs').connect("myIpAddress", ["myCollection"]);
db.myCollection.find({}).forEach(function(err, doc) {
    console.log(doc);
});

EDIT 2:

I'm not sure whether it's related but every now and then I get a "TCP Out of Order" packet from wireshark.

EDIT 3:

I tried to adjust the batch size, this did not work. I also tried MongoSkin to make sure it wasn't mongojs doing something wrong, but that didn't solve anything as well.

Upvotes: 0

Views: 1198

Answers (2)

Ruben Homs
Ruben Homs

Reputation: 591

I fixed it by copying the MongoDB server to the same place where my Node server was running. I think it had to do with the VPN I was connected to, although I can't say that with certainty.

Upvotes: 1

Sachin
Sachin

Reputation: 925

Some other ideas:

  1. Mongoose works pretty well IMO, can you replace Mongojs with Mongoose?

  2. Play with the parameters of the find query and chain a .batch_size() with a high value.

  3. Run Wireshark and monitor the connection and see which interaction is waiting 60 seconds. Might be some peculiar Nodejs - VPN network interaction issue.

Could you post your code?

...edit - try this (havent tested it so might have some syntax quirks - check with your program)

/* Get all the documents (about 1000) */
var db = require('mongojs').connect("myIpAddress", ["myCollection"]);
db.myCollection.find({}).forEach(function(err, doc) {
    console.log(doc);
}).batch_size(2000);

Upvotes: 1

Related Questions