CODEWITHSUNDEEP
pythonherokuflaskredis
MadRabbit
MadRabbit

Reputation: 2520

How to execute function after returning Flask response (hosted on Heroku)?

While I've heared I need to use something called "job queue" I'm new to this and I've had hard time setting it all up. How do I execute function after returning a response in flask? Could you walk me through the process?

Upvotes: 3

Views: 1751

Answers (2)

Jop Knoppers
Jop Knoppers

Reputation: 714

This might be a little to late for you. But for other who have the same problem this might be an easy solution:

import queue import threading import time from flask import Flask from flask import jsonify from flask_restful import Resource, Api

def putting_thread(q):
    while True:
        x = q.get()
        time.sleep(5)
        print(x)

class make:
    def nieuw(self, queue):
        app = Flask(__name__)
        api = Api(app)

        api.add_resource(instructor, '/<action>', resource_class_kwargs={'queue': queue})

        return app

class instructor(Resource):
    def __init__(self, queue):
        self.queue = queue

    def get(self, action = False):
        self.queue.put(action)
        return jsonify(
            status=0
        )

q = queue.Queue()
t = threading.Thread(target = putting_thread, args = (q,), daemon = True)
t.start()

if __name__ == '__main__':
    app = make().nieuw(q)
    app.run(host='127.0.0.1', port=8081, debug=True)

You can test it with: http://127.0.0.1:8081/sometesttext

Upvotes: 0

MadRabbit
MadRabbit

Reputation: 2520

So what I've figured out is that that it's extremely easy to do and even easier on heroku the problem is the documentation is quite scattered around and for someone who's just discovering job queues it may be overwhelming.

For this example I'm going to use Reddis To Go addon on Heroku so the first thing you have to do is install it from your dashboard. After that you set up your flask app too look something like this:

from flask import Flask
from rq import Queue
from redis import Redis
import os
import urllib.parse as urlparse

app = Flask(__name__)

def function_to_queue():
    return "finished"
# Tell RQ what Redis connection to use and parse url from the global variable that was added by the addon
redis_url = os.getenv('REDISTOGO_URL')
urlparse.uses_netloc.append('redis')
url = urlparse.urlparse(redis_url)
conn = Redis(host=url.hostname, port=url.port, db=0, password=url.password)
q = Queue(connection=conn)  #no args implies the default queue

@app.route('/')
def hello():
    ob = q.enqueue(function_to_queue) #Add previously defined function to queue
    return "k?"
if __name__ == '__main__':
    app.run()

Next you have to create python script called run-worker.py with the code below:

import os
import urllib.parse as urlparse
from redis import Redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDISTOGO_URL')
if not redis_url:
    raise RuntimeError('Set up Redis To Go first.')

urlparse.uses_netloc.append('redis')
url = urlparse.urlparse(redis_url)
conn = Redis(host=url.hostname, port=url.port, db=0, password=url.password)

with Connection(conn):
    worker = Worker(map(Queue, listen))
    worker.work()

Now just modify your Procfile on heroku to look like this:

web: gunicorn hello:app --log-file -
worker: python -u run-worker.py

Deploy this, make sure you have both worker and the app started... annnd you're done. Hope this helps others understanding job queueing faster.

Upvotes: 3

Related Questions