CODEWITHSUNDEEP
javascriptpythonnode.jsapi
Matheus Leonardo
Matheus Leonardo

Reputation: 23

Async requests over an API with request rate limiter

I'm working in a project where I need to make requests over an API. The requests return data about a support ticket, but the problem is that i have about 500 tickets to get data about and each one requires one request. To speed up the requests, i tried to build a async routine that generate many requests at the same time. But, since the API that i'm integrating with has a rate limiter of 10 requests per second, some of the routines get the answer "Limit Exceed". If I make the requests sequentially, it's take about 5 minutes.

That way, someone has a tip for me in that task? I tried some solutions like rate-limiter of NodeJS, but it just generate 10 requests simultaneously, and didn't give any kind of error treatment or retry if the request fail.

About the language, it not have restriction, the project is written in NodeJS but have some python code too and didn't have problem to integrate another language.

Upvotes: 2

Views: 2456

Answers (1)

samanime
samanime

Reputation: 26625

Something like this isn't too difficult to create yourself, and it'd give you the flexibility you need.

There are fancy ways like tracking the start and completion time of each and checking if you've sent 10 in the second.

The system probably also limits it to 10 active requests going (i.e., you can't spin up 100 requests, 10 each second, and let them all process).

If you assume this, I'd say launch 10 all at once, then let them complete, then launch the next batch. You could also launch 10, then start 1 additional each time one finishes. You could think of this like a "thread pool".

You can easily track this with a simple variable tracking how many calls are going. Then, just check how many calls are going once a second (to avoid the 1 second limit) and if you have available "threads", fire off that many more new requests.

It could look something like this:

const threadLimit = 10;
const rateLimit = 1000; // ms
let activeThreads = 0;

const calls = new Array(100).fill(1).map((_, index) => index); // create an array 0 through 99 just for an example

function run() {
  if (calls.length == 0) {
    console.log('complete');
    return;
  }

  // threadLimit - activeThreads is how many new threads we can start
  for (let i = 0; i < threadLimit - activeThreads && calls.length > 0; i++) {
    activeThreads++; // add a thread
    call(calls.shift())
      .then(done);
  }
  
  setInterval(run, rateLimit);
}

function done(val) {
  console.log(`Done ${val}`);
  activeThreads--; // remove a thread
}

function call(val) {
  console.log(`Starting ${val}`);
  return new Promise(resolve => waitToFinish(resolve, val));
}


// random function to simulate a network call
function waitToFinish(resolve, val) {
  const done = Math.random() < .1; // 10% chance to finish
  done && resolve(val)
  
  if (!done) setInterval(() => waitToFinish(resolve, val), 10);
  
  return done;
}

run();

Basically, run() just starts up however many new threads it can, based on the limit and how many are done. Then, it just repeats the process every second, adding new ones as it can.

You might need to play with the threadLimit and rateLimit values, as most rate limiting systems don't actually let you go up right to the limit and don't release it as soon as it's done.

Upvotes: 1

Related Questions