CODEWITHSUNDEEP
node.jsrestapireactjs
PurplePanda
PurplePanda

Reputation: 673

Best way to request API and store every minute

I have an app that is hitting the rate limit for an API which is hurting the user experience. I have an idea to solve this but have no idea if this is what should be done ideally to solve this issue. Does this idea makes sense and is it a good way to solve this issue? And how should I go about implementing it? I'm using react-native and nodejs.

Here is the idea:

My app will request the data from a "middleman" API that I make. The middle man API will request data once per minute from the main API that I am having the rate limit problem with (this should solve the rate limit issue then) then store it for the one minute until it updates again. I was thinking the best way to do this is spin a server on AWS that requests from the other API every minute (Is this the easiest way to get a request every minute?) then store it on either a blank middleman webpage (or do I need to store it in a database like MongoDB?). Then my app will call from that middleman webpage/API.

Upvotes: 0

Views: 609

Answers (1)

Salketer
Salketer

Reputation: 15711

Your idea is good.

Your middleman would be a caching proxy. It would act just as you stated. Hava a look at https://github.com/active-video/caching-proxy it does almost what you want. It creates a server that will receive requests of URLs, fetch and cache those, and serve the cached version from now on.

The only downside is that it does not have a lifetime option for the cache. You could either fork to add the option, or run a daemon that would delete the files that are too old to force a re-fetch.

EDIT: A very interesting addition to the caching-proxy would be to have a head request to know if the result changed. While this is not provided by all API, this could become useful if yours is displaying such info. Only if HEAD requests do not count toward your API limits...

Upvotes: 2

Related Questions