CODEWITHSUNDEEP
node.jsasync-awaitpromise
L. Lei
L. Lei

Reputation: 41

Most efficient way to loop through array elements that need to be processed asynchronously

I have 3 async functions. The output of the first is the input of the second. And the output of the first two are the input of the third. Now these 3 will be inside a loop, depending on the length of my array. What is an efficient way to iterate through the array and execute the functions quickly in such a way that the output of the first two functions are still connected to each other? Meaning, they have to have the same length as the array, and their indices are paired with each other. Note that the output does not need to be in order, the ith element of first array just need to pair with ith element of the second array.

For example:

let firstFunctionOutput = [];
let secondFunctionOutput = [];
let thirdFunctionInput = [];
// arr.length = 10,000
// at the end, firstFunctionOutput and secondFunctionOutput should have length = 10,000 as well
for (i < 0; i < arr.length; i++) {
  firstFunctionOutput.push(await firstFunction(arr[i]));
  secondFunctionOutput.push(await secondFunction(arr[i], firstFunctionOutput[i]));
  thirdFunctionInput.push(firstFunctionOutput[i], secondFunctionOutput[i]);
}
await thirdFunction(thirdFunctionInput);

I just feel like this is too slow because we're iterating and there's too much awaits. Any suggestion will be appreciated.

Upvotes: 0

Views: 230

Answers (1)

Teneff
Teneff

Reputation: 32158

Given the above code snipped the thirdFunction needs all of the outputs from firstFunction and secondFunction, but their invocations can be in parallel.

So if each execution of first and second function takes a second the third function could be executed in about 2 seconds

  const firstAndSecond = async (arri) => {
    const output = await firstFunction(arri)
    return [output, await secondFunction(arri, output)]
  }
  thirdFunctionInput = (await Promise.all(arr.map(firstAndSecond))).flat()
  return await thirdFunction(thirdFunctionInput);

TS playground

Upvotes: 2

Related Questions