CODEWITHSUNDEEP
node.jsmongodb
Monkey Boson
Monkey Boson

Reputation: 1282

Mongo's Cursor.nextObject Sometimes Erroneously Returns Null?

I'm combining async.queue and Cursor.nextObject to iterate over a cursor and do some asynchronous work against the returned documents.

There's a great little package that already does this, https://www.npmjs.org/package/mongo-cursor-processing, but it unfortunately doesn't expose the underlying queue, which I need.

So, I tried implementing it myself, but have hit a snag. Sometimes, Cursor.nextObject returns null when in fact there are more documents.

Here's a little code snippet that I've attached to the queue to illustrate:

if (this.cursor && this.length() < this.concurrency) {
    this.cursor.nextObject(function(err, item) {
        console.log(this.name + ': ' + (item ? item._id : '<null>') + ' ' + (err ? err : '<null>'));
        if (item) {
            this.push(item);
        } else {
            // delete this.cursor;
        }
    }.bind(this));
}

The console log shows:

... Maybe 100 lines ...
prop-queue: 511abbd59c0d972a3e000119 <none>
prop-queue: 511abbd59c0d972a3e00011d <none>
prop-queue: 511abbd59c0d972a3e000120 <none>
prop-queue: 511abbd59c0d972a3e000122 <none>
prop-queue: <none> <none>
prop-queue: 511abbd59c0d972a3e000125 <none>
prop-queue: 511abbd59c0d972a3e000127 <none>
prop-queue: 511abbd59c0d972a3e000129 <none>
prop-queue: 511abbd59c0d972a3e00012c <none>
... 1000's more lines before the next null ...

Sometimes, the <none> <none> line is repeated twice before the next call succeeds.

The really interesting part is when I execute the query in the Mongo shell, there's a pause in between when 511abbd59c0d972a3e000122 and 511abbd59c0d972a3e000125 are printed to console. The pause lasts about 0.75s, and is right where the null document is hit in the cursor. I've iterated over thousands of documents in the query, and it's the only pause I experienced. Furthermore, examining the two documents on either side of the null shows no peculiarities.

Any ideas what could cause the two likely-related phenomena?

Upvotes: 2

Views: 745

Answers (1)

Monkey Boson
Monkey Boson

Reputation: 1282

I'm still not sure what's causing the pause, but it seems that is the culprit.

During the pause, Cursor.nextObject is getting called several times before the first returns. Some of these calls are returning null. The solution is to make sure Cursor.nextObject is never called concurrently.

if (this.cursor && !this.cursor_exec && this.length() < this.concurrency) {
    this.cursor_exec = true;
    this.cursor.nextObject(function(err, item) {
        console.log(this.name + ': ' + (item ? item._id : null) + ' ' + (err ? err : null));
        this.cursor_exec = false;
        if (item) {
            this.push(item);
        } else {
            delete this.cursor;
        }
    }.bind(this));
}

Upvotes: 1

Related Questions