CODEWITHSUNDEEP
javascriptasynchronousecmascript-6generator
shinzou
shinzou

Reputation: 6192

Generator function with yield Promise.all()

I want to run a function in chunks so it will wait for 10k promises to resolve and then continue, I use the following generator function: 

  function* processNodes(nodes, task){
    let i;
    let cnt = 0;
    let promiseArray = new Array(10000);
    let pInd = 0;
    let currId;

    for(i = 0; i<nodes.length; i++){
      currId = nodes[i];
      promiseArray[pInd++] = asyncFunc(currId, 0, task); // return a promise
      cnt++;

      if(cnt > 10000){
        console.log("going to yield", promiseArray)
        let pall = Promise.all(promiseArray);
        console.log("promise all", pall);
        yield pall;
        console.log("return from yield");  // never get here
        pInd = cnt = 0;
      }
    }
  }

But it never returns from the yield even though I see that pall is resolved.

Is it possible to do something like this with generator functions?

Edit: I think what I'm trying to do is to implement something like Bluebird's coroutine: http://bluebirdjs.com/docs/api/promise.coroutine.html

Edit2: this is how I call this function:

let x = processNodes(allNodes, someSimpleTask);
x.next();

Upvotes: 3

Views: 10574

Answers (1)

Icepickle
Icepickle

Reputation: 12796

Your problem is that you don't use the generator function as it should be. You are never getting to the console.log('return from field') because at the yield, the code stops executing after the yield statement. Only when you call the iterator again, it will continue after the yield statement (until the next yield statement)

So the generator creates an iterator, that has a value and a bool flag done. As long as the done is not set to true, you can/should call the next function again

A simplified version of your code can be the following

// a very basic async function, just outputting the argument each 5 ms
function asyncFunc(arg) {
  return new Promise(function(resolve) {
    setTimeout(() => {
      console.log(arg);
      resolve();
    }, 5);
  });
}

// the generator
function* generator(processNodes, task) {
  var limit = 4,
    queue = [];
  for (let i = 0; i < processNodes.length; i++) {
    queue.push(task(processNodes[i]));
    if (queue.length >= limit) {
      yield Promise.all(queue);
      // clears the queue after pushing
      console.log('after queue');
      queue = [];
    }
  }
  // make sure the receiver gets the full queue :)
  if (queue.length !== 0) {
    yield Promise.all(queue);
  }
}

function runThroughArguments(args, task) {
  return new Promise(function(resolve) {
    setTimeout(() => {
      var nodes = generator(args, task),
        iterator = nodes.next();

      if (!iterator.done) {
        // if it's not done, we have to recall the functionallity
        iterator.value.then(function q() {
          setTimeout(() => {
            iterator = nodes.next();
            if (!iterator.done && iterator.value) {
              // call the named function (in this case called q) which is this function after the promise.all([]) completed
              iterator.value.then(q);
            } else {
              // everything finished and all promises are through
              resolve();
            }
          }, 2);
        });
      } else {
        iterator.value.then(resolve);
      }
    }, 2);
  });
}

runThroughArguments(
  ['hey', 'you', 'the', 'rock', 'steady', 'crew'], 
  asyncFunc).then(() => console.log('completed'));

console.log('runs before everything');

In the above snippet, it runs through a promise as well. So than you can be notified when the full queue is through, it is a bit more complex than the original snippet which can be found here

A more comprehensible explanation of the patterns you are using can be found on MDN

Upvotes: 3

Related Questions