CODEWITHSUNDEEP
pythonapiflaskpython-requestsbig-o
limaBEAN
limaBEAN

Reputation: 67

Improve latency of HTTP request function in Python using Flask

In the process of using Eventbrite's API to power a website with their events, however, I am stuck on how to implement project.

Currently, I am wanting to filter events via dates and show them on the front page, however, Eventbrite API limits to only 50 items per page. The API request pushes out 86 pages with 50 items each.

How can I go about presenting those events at once while lowering latency? What I currently have takes 104 seconds to respond and render results to page. I'm scratching the surface on Big O Notation theory, however, I think I'm using O(n²).

here's the code:

routes.py:

from flask import Flask, render_template
from eventbrite import Eventbrite
from datetime import datetime

import requests, json

app = Flask(__name__)


# TODO: Remove outh key in file and register in sys
oKey = ‘<oauth-key>’

o_token_url = '&token={0}'.format(oKey)

@app.route("/")
def index():

    base_url = 'https://www.eventbriteapi.com/v3/'
    regions_url = base_url + 'system/regions/?token={0}'.format(oKey)
    regions_result = requests.get(regions_url)
    regions_obj = regions_result.json()

    e_url = base_url + 'events/search/?token={0}&location.address=<city>&start_date.range_start=2018-07-02T12%3A00%3A00Z&page=2'.format(oKey)
    e_result = requests.get(e_url)
    e_obj = e_result.json()

   list_events = []

    for pg in range(1, e_obj['pagination']['page_count']):
        print(pg)
        startTime = datetime.now()

        e_url = base_url + 'events/search/?token={0}&location.address=<city>&start_date.range_start=2018-07-02T12%3A00%3A00Z&page={1}'.format(oKey, pg)
        e_result = requests.get(e_url)
        e_obj = e_result.json()

        for i in e_obj['events']:
            an_event = {}
            an_event['name'] = i['name']['text']
            list_events.append(htx_event)

    print(datetime.now() - startTime)
    return render_template('index.html', events=list_events)

if __name__ == "__main__":
    app.run(host='0.0.0.0')

index.html:

{% for event in events %}
{{event['name']}}<br>

{% endfor %}

I am aware of EB API batched requests endpoint however, I can't wrap my head around on how to send the list of url dictionaries with requests.

Any tips on how to optimize the code above would be appreciated. I figured I can limit the amount of events that are shown at any given time, however, interested in how others would approach this issue.

Upvotes: 1

Views: 3510

Answers (1)

Fine
Fine

Reputation: 2154

It takes long time not because it's a CPU intensive operation (which could be solved with algorithm optimization), but because each HTTP request takes time. To mitigate that, you can make requests concurrent (i.e. simultaneous). To further reduce latency, you can tune re-use of TCP connections so you wouldn't waste time to create a TCP connection for each HTTP request.

Upvotes: 3

Related Questions