Recursively create new promises

Question

I'm trying to get the Promise.all function to work with a recursively structure.

The function that's been called from the for loop can create promises as well, but somehow the Promise.all function seems to be finished before the work has finished, and I don't have a clue why.

this.promised = [];
parseItem( relation ) {

    if( relation.wasResolved === true ) {
        return Promise.resolve();
    }

    return new Promise(async (resolve) => {

        console.log( 'Parsing ' + relation.displaytitle );

        let result = await this.MediaWiki.ask('[[Has source element::'+ relation.fulltext +']]|?Has target element');

        // Loop over the API results
        for( let k in result ) {

            let targetElements = result[k].printouts['Has target element'];

            // Loop over every target element
            for( let index in targetElements) {

                let itemIndex = this.getIndex( targetElements[index] );

                // Only resolve known elements
                if( itemIndex !== -1 ) {
                    // Set parent && resolve childs
                    this.elementsInView[itemIndex].parent = relation;
                    // Resolve
                    this.promised.push( this.parseItem( this.elementsInView[itemIndex] ) );
                }
            }
        }

        // Set wasResolved to true
        relation.wasResolved = true;

        resolve(true);
    });
}

update() {

    // Loop over every element in the view
    for( let index in this.elementsInView ) {
        let element = this.elementsInView[index];
        // If the viewItem has a assigned hostId, add it to the queue
        if( element.host !== -1 ) {
            this.promised.push( this.parseItem( element ) );
        }
    }

    Promise.all(this.promised).then(value =>{
        this.finalize();
    });

}

finalize() {
    for( let index in this.elementsInView ) {
        let element = this.elementsInView[index];
        if( element.wasResolved === true ) {
            console.log( element );
        }
    }
}

Answer 1

The problem comes from you trying to share this.promised across update() and the async behaviour inside parseItem().

To fix this, you can instead return a Promise from parseItem() that represents every Promise created inside.

For example, a simplified version of this:

async parseItem(relation) {
  const result = await ask(relation);
  return await Promise.all(result.map(i => {
    const relations = i.relations;
    return Promise.all(relations.map(parseItem));
  }));
}

As you can see, inside parseItem() we're waiting for everything to recurse before resolving the Promise.

To apply this to your code, it might look something like this:

async parseItem(relation) {
  if (relation.wasResolved === true) {
    return true;
  }

  console.log('Parsing ' + relation.displaytitle);

  const result = await this.MediaWiki.ask('[[Has source element::' + relation.fulltext + ']]|?Has target element');

  const nestedPromises = result.map(el => {
    const targetElements = el.printouts['Has target element'];

    // This returns an array of an array of promises
    return targetElements
      .map(tel => this.getIndex(tel))
      .filter(index => index !== -1)
      .map(index => {
        this.elementsInView[index].parent = relation;

        // This returns a promise
        return this.parseItem(this.elementsInView[index]);
      });
  });

  // Easy way to flatten array of arrays
  const promises = [].concat.apply([], nestedPromises);
  await Promise.all(promises);

  // Set wasResolved to true
  relation.wasResolved = true;

  return true;
}

I can't guarantee this will work, though. It depends on the nature of this data structure and what effect it has to modify it whilst resolving all of these promises.

The main issue that raises questions is this line:

this.elementsInView[index].parent = relation;

What effect does this have on parseItem? And as a result of this mutation, I'd now be concerned whether or not multiple items passed to parseItem by update that are running in an interleaving manner might be simultaneously modifying data that they depend on.

At least in terms of the original question, this method should assist you in waiting for all promises to finish.