Why is it not waiting for an awaited Promise.all to resolve?

Question

I previously had working code that inefficiently called awaits on every element of an iterable. I'm in the process of refactoring to use Promise.All. However, my code is not waiting for the Promise.All to resolve before executive further code.

Specficially, the purgeRequestPromises line executes before the initial Promise.All resolves. I'm not sure why that is? retrieveSurrogateKey is an async function, so its return line will be wrapped in a resolved promise.

try {
    //retrieve surrogate key associated with each URL/file updated in push to S3
    const surrogateKeyPromises = urlArray.map(url => this.retrieveSurrogateKey(url));
    const surrogateKeyArray = await Promise.all(surrogateKeyPromises).catch(console.log);

    //purge each surrogate key
     const purgeRequestPromises = surrogateKeyArray.map(surrogateKey => this.requestPurgeOfSurrogateKey(surrogateKey));
     await Promise.all(purgeRequestPromises);

     // GET request the URLs to warm cache for our users
     const warmCachePromises = urlArray.map(url => this.warmCache(url));
     await Promise.all(warmCachePromises)
} catch (error) {
    logger.save(`${'(prod)'.padEnd(15)}error in purge cache: ${error}`);
    throw error
} 

async retrieveSurrogateKey(url) {
    try {
        axios({
            method: 'HEAD',
            url: url,
            headers: headers,
        }).then(response => {
            console.log("this is the response status: ", response.status)
            if (response.status === 200) {
                console.log("this is the surrogate key!! ", response.headers['surrogate-key'])
                return response.headers['surrogate-key'];
            }

        });
    } catch (error) {
        logger.save(`${'(prod)'.padEnd(15)}error in retrieveSurrogateKey: ${error}`);
        throw error
    }
}

I know that the purgeRequestPromises executes early, because I get errors complaining that I've set my Surrogate-Key header as undefined in my HEAD request:

async requestPurgeOfSurrogateKey(surrogateKey) {
    headers['Surrogate-Key'] = surrogateKey

    try {
        axios({
                method: `POST`,
                url: `https://api.fastly.com/service/${fastlyServiceId}/purge/${surrogateKey}`,
                path: `/service/${fastlyServiceId}/purge${surrogateKey}`,
                headers: headers,
            })
            .then(response => {
                console.log("the status code for purging!! ", response.status)
                if (response.status === 200) {
                    return true
                }
            });
    } catch (error) {
        logger.save(`${'(prod)'.padEnd(15)}error in requestPurgeOfSurrogateKey: ${error}`);
        throw error;
    }
}

Answer 1

retrieveSurrogateKey is synchronously returning undefined: the value in the try block is a promise and no errors are thrown synchronously, so the catch clause is never executed and execution falls out the bottom, returning undefined from the function body.

You could try something like:

function retrieveSurrogateKey(url) {  // returns a promise
    return axios({
//  ^^^^^^
        method: 'HEAD',
        url: url,
        headers: headers,
    }).then(response => {
        console.log("this is the response status: ", response.status)
        if (response.status === 200) {
            console.log("this is the surrogate key!! ", response.headers['surrogate-key'])
            return response.headers['surrogate-key'];
        }

    }).catch(error => {
       logger.save(`${'(prod)'.padEnd(15)}error in retrieveSurrogateKey: ${error}`);
       throw error;
    });
}

Note that it is superfluous to declare a function returning a promise as async if it doesn't use await. There is also a secondary problem in this line:

const surrogateKeyArray = await Promise.all(surrogateKeyPromises).catch(console.log);

The catch clause will will fulfill the promise chain unless the error is rethrown. You could (perhaps) leave off the .catch clause or recode it as

.catch( err=> { console.log(err); throw err} );

Answer 2

You don't have to remove async from retrieveSurrogateKey() in order for it to work. In fact it's more readable if you don't. As was already explained, the problem is that the promise returned by retrieveSurrogateKey() does not follow the completion of the promise returned by the call to axios(). You need to await it:

async retrieveSurrogateKey(url) {
  try {
    const response = await axios({
      method: 'HEAD',
      url,
      headers,
    });

    console.log('this is the response status: ', response.status);

    if (response.status === 200) {
      const surrogateKey = response.headers['surrogate-key'];
      console.log('this is the surrogate key!! ', surrogateKey);
      return surrogateKey;
    }
  } catch (error) {
    logger.save(`${'(prod)'.padEnd(15)}error in retrieveSurrogateKey: ${error}`);
    throw error;
  }
}

This preserves the same logic you currently have, but you'll notice that when response.status !== 200, you end up with a resolved promise of undefined, rather than a rejected promise. You might want to use validateStatus to assert the exact status of 200. By default axios resolves any response with a status >= 200 and < 300:

async retrieveSurrogateKey(url) {
  try {
    const response = await axios({
      method: 'HEAD',
      url,
      headers,
      validateStatus(status) {
        return status === 200;
      } 
    });
    
    const surrogateKey = response.headers['surrogate-key'];
    console.log('this is the surrogate key!! ', surrogateKey);
    return surrogateKey;
  } catch (error) {
    logger.save(`${'(prod)'.padEnd(15)}error in retrieveSurrogateKey: ${error}`);
    throw error;
  }
}

This way, you're always guaranteed a surrogate key, or a rejected promise.