CODEWITHSUNDEEP
javascriptnode.jstypescriptaxios
cbor
cbor

Reputation: 41

Setting delay/timeout for axios requests in map() function

I am using node and axios (with TS, but that's not too important) to query an API. I have a suite of scripts that make calls to different endpoints and log the data (sometimes filtering it.) These scripts are used for debugging purposes. I am trying to make these scripts "better" by adding a delay between requests so that I don't "blow up" the API, especially when I have a large array I'm trying to pass. So basically I want it to make a GET request and pause for a certain amount of time before making the next request.

I have played with trying setTimeout() functions, but I'm only putting them in places where they add the delay after the requests have executed; everywhere I have inserted the function has had this result. I understand why I am getting this result, I just had to try everything I could to at least increase my understanding of how things are working.

I have though about trying to set up a queue or trying to use interceptors, but I think I might be "straying far" from a simpler solution with those ideas.

Additionally, I have another "base script" that I wrote on the fly (sorta the birth point for this batch of scripts) that I constructed with a for loop instead of the map() function and promise.all. I have played with trying to set the delay in that script as well, but I didn't get anywhere helpful.

var axios = require('axios');
var fs = require('fs');

const Ids = [arrayOfIds];

try {
    // Promise All takes an array of promises
   Promise.all(Ids.map(id => {
        // Return each request as its individual promise
        return axios 
          .get(URL + 'endPoint/' + id, config)
    }))

    .then((vals) =>{
        // Vals is the array of data from the resolved promise all
        fs.appendFileSync(`${__dirname}/*responseOutput.txt`, 
vals.map((v) => {
            return `${JSON.stringify(v.data)} \n \r`
        }).toString())
    }).catch((e) => console.log)
} catch (err) {
  console.log(err);
}

No errors with the above code; just can't figure out how to put the delay in correctly.

Upvotes: 2

Views: 1681

Answers (1)

Rick Su
Rick Su

Reputation: 16440

You could try Promise.map from bluebird

It has the option of setting concurrency

var axios = require('axios');
var fs = require('fs');
var Promise = require('bluebird');

const Ids = [arrayOfIds];

let concurrency = 3;  // only maximum 3 HTTP request will run concurrently

try {

    Promise.map(Ids, id => {

        console.log(`starting request`, id);

        return axios.get(URL + 'endPoint/' + id, config)

    }, { concurrency })

    .then(vals => {
        console.log({vals});
    })
    ;

} catch (err) {
  console.log(err);
}

Upvotes: 1

Related Questions