CODEWITHSUNDEEP
python
imagineerThat
imagineerThat

Reputation: 5553

How to politely make massive amounts of api calls?

I have a script that takes about 15 minutes to run. It basically hits a bunch of api endpoints. Because some of our production services run on where these endpoints are hosted, I want to throttle my script (which runs weekly) so it won't impact our systems as much. What's the easiest way to do this? The script is mostly for loops. Should I just add some sleep statements or is there a more standardized way to do this? I don't care if the script takes a long time to run, because I'll run it overnight. 

for i in list: <-big list 
   make_call_to_api() <-typically couple seconds
   time.sleep(5) <-sleep 5 seconds

Upvotes: 2

Views: 124

Answers (1)

Andriy Ivaneyko
Andriy Ivaneyko

Reputation: 22021

I think using time is fine. However alternatively you can use threading.Timer:

from threading import Timer

def make_api_with_interval(*arg, **kwargs):
    # BODY of make api call
    make_call_to_api()
    Timer(30.0, make_api_with_interval, ["args"], {'kwarg': 'kwarg'}).start()

Timer(30.0, make_api_with_interval).start() # after 30 seconds, api call would be performed.

Upvotes: 1

Related Questions