CODEWITHSUNDEEP
ajaxdjango
Shezmula
Shezmula

Reputation: 47

Django - How to make API calls at constant intervals

What I want to do: I have built a dashboard which shows Key Performance Indicators that I get from an API at Zendesk.com. It needs to update at regular intervals (I am trying to use time.sleep() to accomplish this) but it cannot be too often as I have a 700 request per minute limit. I used Django so I could create an interface to show the data. IT WILL NEVER BE USED AS A WEBSITE.

The problem: I use jquery "get" calls to get data from my script. If I use "python manage.py runserver" it eventually creates what I am guessing are many many workers and makes calls to my script far far too often and blows past my 700 request limit with Zendesk. If I use "python manage.py runserver --nothreading" it works great HOWEVER when I click a button in the navbar or try to refresh the page, I have to wait for my script to run fully before it gets to the task.

Relevant code of my program:

As an example, in the html code I call data from json:

     <script type="text/javascript">
         $(document).ready(function() {
           function pollSiteData () {
           $.get('/nb_new/', function(resp) {
             $('#nb_new').text(resp['nb_new']);
           });
           window.setTimeout(pollSiteData,10000);
           }
           pollSiteData();
           });
     </script>

Which calls into a class based view:

url(r'^nb_new/',views.SiteNumberView.as_view())

The view is as follows(the xxxxxx is my login information for testing purposes):

class SiteNumberView(views.APIView):
    def get(self, request, *args, **kwargs):
        data = engine.main("xxxxx","xxxxxx","xxxxxx")
        return JsonResponse(data)

The above class based view calls the function "main" from a file engine.py that I made.

The function is:

def main(*args):
    client = ZenClient(args)
    time.sleep(45)
    print("Next iteration")
    return get_infos(client, "open", "pending", "new")

the Main function call get_infos which returns a dictonary that I convert to json.

def get_infos(client, status, status2, status3):

    #Script that downloads stats from Zendesk

    return {"nb_new": str(len(filtered_new_tickets))}

When I run "python manage.py runserver" we can see that it is using my script far too often:

Next iteration
Next iteration
Next iteration
[09/May/2018 08:01:21] "GET /nb_new/ HTTP/1.1" 200 86
Next iteration
[09/May/2018 08:01:30] "GET /nb_new/ HTTP/1.1" 200 86
Next iteration
[09/May/2018 08:01:35] "GET /nb_new/ HTTP/1.1" 200 86
Next iteration
[09/May/2018 08:01:51] "GET /nb_new/ HTTP/1.1" 200 86
[09/May/2018 08:02:04] "GET /nb_new/ HTTP/1.1" 200 86
Next iteration

When I use "python manage.py runserver --nothreading" it makes calls at good intervals but buttons on the interface are non responsive. (I imagine because it has to finish the script before it can acknowledge a button press.)

Here is a picture of my interface ("nb_new" updates the number of tickets next to "support needs response")

Interface

Conclusion/Question: How can I make sure that my script is only running lets say once every minute and 15 seconds but still let the interface respond to a button push or refresh the page? Please try to avoid giving solutions that would require me to use an entirely different framework. Time is a very scarce resource unfortunately.

Upvotes: 0

Views: 1873

Answers (1)

xyres
xyres

Reputation: 21734

Here's a simple solution:

  1. Create a separate Python script dedicated to ping Zendesk API at regular intervals. When the new data arrives, it will put the data in a queue or a database.
  2. Create a Django view that would respond to your jQuery calls. It would read the data from the database or queue and return it to the jQuery request. It will not actually make the requests to Zendesk API itself.

The easier way to store the new updates will be to create a dedicated model just for this data. Create one instance of this model and just keep updating its state (field values) whenever new data arrives instead of creating a separate object everytime. Then from you view, whenever you get a jQuery call, you can return this object.

If you don't like using this model-and-database approach, you can use a memory based database (queue) to store the data from your script and then access it from your Django view. I recommend Redis for this.

Example script:

# start_server.py

import subprocess
import threading
import time


def ping():
    while True:
        print("Pinging ...") # do actual API pinging stuff here
        time.sleep(10)

t = threading.Thread(target=ping)
t.start() # this will run the `ping` function in a separate thread

# now start the django server
subprocess.call(['python', 'manage.py', 'runserver'])

Now, all you have to do is run python start_server.py and it will automatically run the Django server as well as the API ping function.

Upvotes: 1

Related Questions