CODEWITHSUNDEEP
javascriptnode.js
lyonTypescript
lyonTypescript

Reputation: 135

How can I get all data from a JSON api when pagination is enabled?

I need to get all data from this api: https://hn.algolia.com/api/v1/search_by_date?query=nodejs, but I'm getting the first page(page 1) data with this code:

urlApi = 'https://hn.algolia.com/api/v1/search_by_date?query=nodejs'

let repo = await axios.get(urlApi);
repo.data.hits.map((data) => {
    console.log(data);
 });

This JSON contains 'nbPages': 50, But I need to map all 50 JSON pages.

Do you have any ideas for mapping?

Upvotes: 1

Views: 16294

Answers (2)

Klaycon
Klaycon

Reputation: 11080

You could use a while loop and just append the hits to an array until you've completed all requests:

let repo = null, page = 0, results = [];
do {
    repo = await axios.get(`${urlApi}&page=${page++}`);
    results = results.concat(repo.data.hits);
} while(repo.data.page < repo.data.nbPages)

console.log(results);

Upvotes: 4

Updater
Updater

Reputation: 459

Due to the fact that the standard value for "hitsPerPage" is 20, you can set this value to get all of the hits at once by using this URL instead (20 hits * 50 pages = 1000):

https://hn.algolia.com/api/v1/search_by_date?query=nodejs&hitsPerPage=1000

Upvotes: 3

Related Questions