CODEWITHSUNDEEP
node.jsasynchronousaxios
GarethPrice
GarethPrice

Reputation: 1104

Async + Axios - Rate control

I am currently sending a GET request to an API using promises 200 times. Not surprisingly, there is a max number of connections allowed within such a short time span.

I am using axios to do this:

const memberPromises = members.map(member => axios.get(`players/%23${member.tag}`))

axios.all().then().catch() // etc

..where members can be up to 200 elements.

There doesn't seem a way to rate-control these requests natively within axios, but how would I use async's (or another library if there is better) queue method with a concurrent parameter to limit the number of simultaneous requests?

Upvotes: 3

Views: 3673

Answers (2)

lord5et
lord5et

Reputation: 500

const axios = require('axios');
const axiosThrottle = require('axios-throttle'); 
//pass axios object and value of the delay between requests in ms
axiosThrottle.init(axios,200)
const options = {
  method: 'GET',
};
const promises = [];
const responseInterceptor = response => {
  console.log(response.data);
  return response;
};

//add interceptor to work with each response seperately when it is resolved
axios.interceptors.response.use(responseInterceptor, error => {
  return Promise.reject(error);
});

for (let index = 0; index < members.length; index++) {
  options.url = `http://yourapiurl/players/%23${member.tag}`;
  promises.push(axiosThrottle.getRequestPromise(options, index));
}

//run when all promises are resolved
axios.all(promises).then(responses => {
  console.log(responses.length);
});

https://github.com/arekgotfryd/axios-throttle

Upvotes: 1

kkpoon
kkpoon

Reputation: 1989

How about this?

map the members to promises, resolve the promise after a timeout

let RATE_LIMIT_BASE = 100; //100ms separation

let promises = members.map((member, idx) => {
    return new Promise((resolve, reject) => {
        setTimeout(
            () =>
                axios
                    .get(`players/%23${member.tag}`)
                    .then(res => resolve(res))
                    .catch(err => reject(err)),
            RATE_LIMIT_BASE * idx
        );
    });
});

Promise.all(promises).then().catch(); // etc;

In comment, as marvel308 said,

this could fail in certain cases, since setTimeout just guarantees that the code will execute atleast 100ms after setTimeout is called, they could still queue up in the event queue in case some CPU intensive task was blocking the call stack

Thanks marvel308 for the advise

Upvotes: 5

Related Questions