How can I fetch an array of URLs with Promise.all?

Question

If I have an array of urls:

var urls = ['1.txt', '2.txt', '3.txt']; // these text files contain "one", "two", "three", respectively.

And I want to build an object that looks like this:

var text = ['one', 'two', 'three'];

I’ve been trying to learn to do this with fetch, which of course returns Promises.

Some things I’ve tried that don’t work:

var promises = urls.map(url => fetch(url));
var texts = [];
Promise.all(promises)
  .then(results => {
     results.forEach(result => result.text()).then(t => texts.push(t))
  })

This doesn’t look right, and in any case it doesn’t work — I don’t end up with an array ['one', 'two', 'three'].

Is using Promise.all the right approach here?

Answer 1

Async/await version

var promises = urls.map(async (url) => {
    let res = await fetch(url)
    return await res.json()
})
const results = await Promise.all(promises)

Answer 2

// Here, you have an array called todos containing three URLs.

const todos = [
  "https://jsonplaceholder.typicode.com/todos/1",
  "https://jsonplaceholder.typicode.com/todos/2",
  "https://jsonplaceholder.typicode.com/todos/3",
];

// asunchronous function to fetch todo of the respective url

const fetchTodo = async (url) => {
  const res = await fetch(url);
  if (!res.ok) {
    throw new Error(`HTTP error! status: ${res.status}`);
  }
  return res.json();
};


// The todos.map((item) => fetchTodo(item)) part creates an array of promises by mapping each URL to the result of the fetchTodo function. Promise.all then waits for all these promises to settle (either resolve or reject).

Promise.all(todos.map((item) => fetchTodo(item)))
  .then((res) => {
    console.log(res);
  })
  .catch((err) => console.error(err));

// fetches JSON data from multiple URLs concurrently using Promise.all,

Answer 3

The following also works for me.

    Promise.all([
      fetch(QUESTIONS_API_BASE_URL).then(res => res.json()),
      fetch(SUBMISSIONS_API_BASE_URL).then(res => res.json())
    ])
    .then(console.log)

Answer 4

Here is a clean way to do it.

const requests = urls.map((url) => fetch(url)); 
const responses = await Promise.all(requests); 
const promises = responses.map((response) => response.text());
return await Promise.all(promises);

Answer 5

In case, if you are using axios. We can achieve this like:

const apiCall = (endpoint:string)=> axios.get(${baseUrl}/${endpoint})

axios.all([apiCall('https://first-endpoint'),apiCall('https://second-endpoint')]).then(response => {
            response.forEach(values => values)
            }).catch(error => {})  

Answer 6

Yes, Promise.all is the right approach, but you actually need it twice if you want to first fetch all urls and then get all texts from them (which again are promises for the body of the response). So you'd need to do

Promise.all(urls.map(u=>fetch(u))).then(responses =>
    Promise.all(responses.map(res => res.text()))
).then(texts => {
    …
})

Your current code is not working because forEach returns nothing (neither an array nor a promise).

Of course you can simplify that and start with getting the body from each response right after the respective fetch promise fulfilled:

Promise.all(urls.map(url =>
    fetch(url).then(resp => resp.text())
)).then(texts => {
    …
})

or the same thing with await:

const texts = await Promise.all(urls.map(async url => {
  const resp = await fetch(url);
  return resp.text();
}));

Answer 7

The suggested array urls = ['1.txt', '2.txt', '3.txt'] does not make much sense to me, so I will instead use:

urls = ['https://jsonplaceholder.typicode.com/todos/2',
        'https://jsonplaceholder.typicode.com/todos/3']

The JSONs of the two URLs:

{"userId":1,"id":2,"title":"quis ut nam facilis et officia qui",
 "completed":false}
{"userId":1,"id":3,"title":"fugiat veniam minus","completed":false}

The goal is to get an array of objects, where each object contains the title value from the corresponding URL.

To make it a little more interesting, I will assume that there is already an array of names that I want the array of URL results (the titles) to be merged with:

namesonly = ['two', 'three']

The desired output is an array of objects:

[{"name":"two","loremipsum":"quis ut nam facilis et officia qui"},
{"name":"three","loremipsum":"fugiat veniam minus"}]

where I have changed the attribute name title to loremipsum.

const namesonly = ['two', 'three'];
const urls = ['https://jsonplaceholder.typicode.com/todos/2',
  'https://jsonplaceholder.typicode.com/todos/3'];

Promise.all(urls.map(url => fetch(url)
  .then(response => response.json())
  .then(responseBody => responseBody.title)))
  .then(titles => {
    const names = namesonly.map(value => ({ name: value }));
    console.log('names: ' + JSON.stringify(names));
    const fakeLatins = titles.map(value => ({ loremipsum: value }));
    console.log('fakeLatins:\n' + JSON.stringify(fakeLatins));
    const result =
      names.map((item, i) => Object.assign({}, item, fakeLatins[i]));
    console.log('result:\n' + JSON.stringify(result));
  })
  .catch(err => {
    console.error('Failed to fetch one or more of these URLs:');
    console.log(urls);
    console.error(err);
  });

.as-console-wrapper { max-height: 100% !important; top: 0; }

Reference

• Convert an array of values to an array of objects

Answer 8

You should use map instead of forEach:

Promise.all(urls.map(url => fetch(url)))
.then(resp => Promise.all( resp.map(r => r.text()) ))
.then(result => {
    // ...
});

Answer 9

For some reason neither of Bergi's examples worked for me. It would simply give me empty results. After some debugging it seemes like the promise would return before the fetch had finished, hence the empty results.

However, Benjamin Gruenbaum had an answer here earlier, but deleted it. His method did work for me, so I'll just copy-paste it here, as an alternative in case anyone else runs into any problems with the first solution here.

var promises = urls.map(url => fetch(url).then(y => y.text()));
Promise.all(promises).then(results => {
    // do something with results.
});