CODEWITHSUNDEEP
javascriptnode.jsasynchronousrate-limitingthrottling
amyspark
amyspark

Reputation: 520

Rate limit API queries in node.js

I am attempting to rate limit queries to the Battle.net API up to their limits of 100 calls/second, 36,000/hour.

My current code looks like this:

var async = require ('async');
var bnet = require ('battlenet-api');

async.eachSeries(regions, function(region, callback) {
    bnet.wow.realmStatusAsync({origin: region}).spread(/*...snip...*/)
});

What I'd need here is to make sure that no calls to battlenet-api run up to the limit, and if so, queue them.

I have looked at timequeue.js, limiter and async's own capabilities and none seems to provide what I am looking for.

Does someone know how to achieve this?

Upvotes: 3

Views: 1805

Answers (1)

dm03514
dm03514

Reputation: 55932

A couple possibilities are:

  1. Wait until you get throttled and backoff your requests
  2. Keep track of your limits and used limits inside of your application.

While i think both cases need to be handled, the second one is more valuable, because it will let your application know at any time how many requests have been made for a given timeframe.

You could focus on implementing this on the hour level and letting the backoff/retry catch the cases of going over the 100 req/s timelimit.

To implement this, your application would have to keep track of how many requests are available for the given time period. Before each request the request count has to be checked to see if there are available requests, if there are the API request can be made and if not then no request will be made and your application will throttle itself. Additionally, the limits need to be reset when appropriate.

There are quite a few articles on using redis to accomplish this, a quick google search should uncover them

Upvotes: 2

Related Questions