CODEWITHSUNDEEP
node.jsmongodbnode-mongodb-native
chovy
chovy

Reputation: 75774

How to get cursor.forEach() iterator to wait for promise chain to resolve?

Using node.js mongodb native module I'm trying to use a promisified iterator but it is not waiting for the promise chain to resolve before firing the finalCallback

db.collection('items').find({}).forEach(promiseIterator, finalCallback);

function promiseIterator(doc){
    return Promise.resolve(doc)
        .then(res => {
            console.log(res); // this fires *after* finalCallback fires
        })
}

function finalCallback(res){
    console.log(res); // this fires *before* promiseIterator resolves all promise chains
}

The doc is here: https://mongodb.github.io/node-mongodb-native/2.2/api/Cursor.html#~resultCallback

Their ES6 examples all use generators so I'm not sure if Promises even work here. But the promise chain is not fully resolved before firing finalCallback.

https://mongodb.github.io/node-mongodb-native/2.2/reference/ecmascript6/crud/

The reason I'm having this issue is because my promiseIterator needs to make several async/promisified calls on each document. (I am unable to load all docs into memory with .toArray() as there are over a million documents and I get a process out of memory error when I try that.

Upvotes: 0

Views: 1862

Answers (2)

antti.mann
antti.mann

Reputation: 11

Unfortunately you cannot. Mongodb native's forEach doesn't wait for Promises to resolve as you noticed. They just call for a callback function and continues to next object.

Cursor's forEach source code (line 769)

http://mongodb.github.io/node-mongodb-native/2.2/api/lib_cursor.js.html

You need to iterate manually with next and hasNext.

http://mongodb.github.io/node-mongodb-native/2.2/api/Cursor.html#next

EDIT: Example added.

You can use promises exactly as jstice4all succested. Here is his example with more details.

db.collection('items').find({}, (err, resultCursor) => {
  resultCursor.next(processItem);

  function processItem(err, item) {
    if(item === null) return; // All done!

    // Add your asynchronous function/chain of functions in place of Promise.resolve()
    Promise.resolve().then(() => {
      resultCursor.next(processItem); // Read next item from database
    });
  }
});

Of course this will process all objects serially, one after another. If you want to process all objects in parallel and wait for them to finish before doing something else, you should use collect all created promises with Promise.all.

btw. nextObject is deprecated, so you can use next instead of it.

Upvotes: 1

jstice4all
jstice4all

Reputation: 1948

You can use a recursion with nextObject instead of forEach:

db.collection('items').find({}, function(err, resultCursor) {
  function processItem(err, item) {
    if(item === null) {
      return; // All done!
    }

    externalAsyncFunction(item, function(err) {
      resultCursor.nextObject(processItem);
    });

  }

  resultCursor.nextObject(processItem);
}

Upvotes: 1

Related Questions