CODEWITHSUNDEEP
pythonflaskget
SFlagg
SFlagg

Reputation: 369

How to loop GETs until a certain response is received

I'm looking for some advice, or a relevant tutorial regarding the following:

My task is to set up a flask route that POSTs to API endpoint X, receives a new endpoint Y in X's response, then GETs from endpoint Y repeatedly until it receives a certain status message in the body of Y's response, and then returns Y's response.

The code below (irrelevant data redacted) accomplishes that goal in, I think, a very stupid way. It returns the appropriate data occasionally, but not reliably. (It times out 60% of the time.) When I console log very thoroughly, it seems as though I have bogged down my server with multiple while loops running constantly, interfering with each other.

I'll also receive this error occasionally:

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /book

import sys, requests, time, json
from flask import Flask, request

# create the Flask app
app = Flask(__name__)

# main booking route
@app.route('/book', methods=['POST']) #GET requests will be blocked
def book():

    # defining the api-endpoints
    PRICING_ENDPOINT = ...

    # data to be sent to api
    data = {...}

    # sending post request and saving response as response object
    try:
        r_pricing = requests.post(url = PRICING_ENDPOINT, data = data)
    except requests.exceptions.RequestException as e:
        return e
        sys.exit(1)

    # extracting response text
    POLL_ENDPOINT = r_pricing.headers['location']

    # setting data for poll
    data_for_poll = {...}

    r_poll = requests.get(POLL_ENDPOINT, data = data_for_poll)

    # poll loop, looking for 'UpdatesComplete'
    j = 1
    poll_json = r_poll.json()
    update_status = poll_json['Status']

    while update_status == 'UpdatesPending':
        time.sleep(2)
        j = float(j) + float(1)
        r_poll = requests.get(POLL_ENDPOINT, data = data_for_poll)
        poll_json = r_poll.json()
        update_status = poll_json['Status']

    return r_poll.text

Upvotes: 0

Views: 305

Answers (2)

Arunmozhi
Arunmozhi

Reputation: 964

This is more of an architectural issue more than a Flask issue. Long-running tasks in Flask views are always a poor design choice. In this case, the route's response is dependent on two endpoints of another server. In effect, apart from carrying the responsibility of your app, you are also carrying the responsibility of another server.

Since the application's design seems to be a proxy for another service, I would recommend creating the proxy in the right way. Just like book() offers the proxy for PRICING_ENDPOINT POST request, create another route for POLL_ENDPOINT GET request and move the polling logic to the client code (JS).

Update: If you cannot for some reason trust the client (browser -> JS) with the POLL_ENDPOINT information in a hidden proxy like situation, then maybe move the polling to a task runner like Celery or Python RQ. Although, it will introduce additional components to your application, it would be the right way to go.

Upvotes: 1

Aleksandr Bakhmach
Aleksandr Bakhmach

Reputation: 154

Probably you get that error because of the HTTP connection time out with your API server that is looping. There are some standards for HTTP time connection and loop took more time that is allowed for the connection. The first (straight) solution is to "play" with Apache configs and increase the HTTP connection time for your wsgi. You can also make a socket connection and in it check the update status and close it while the goal was achieved. Or you can move your logic to the client side.

Upvotes: 0

Related Questions