CODEWITHSUNDEEP
pythonnginxflaskgunicornmod-wsgi
Lonewolf
Lonewolf

Reputation: 527

Gunicorn Flask Caching

I have a Flask application that is running using gunicorn and nginx. But if I change the value in the db, the application fails to update in the browser under some conditions.

I have a flask script that has the following commands

from msldata import app, db, models
path = os.path.dirname(os.path.abspath(__file__))
manager = Manager(app)

@manager.command
def run_dev():
    app.debug = True
    if os.environ.get('PROFILE'):
        from werkzeug.contrib.profiler import ProfilerMiddleware
        app.config['PROFILE'] = True
        app.wsgi_app = ProfilerMiddleware(app.wsgi_app, restrictions=[30])

    if 'LISTEN_PORT' in app.config:
        port = app.config['LISTEN_PORT']
    else:
        port = 5000

    print app.config
    app.run('0.0.0.0', port=port)
    print app.config

@manager.command
def run_server():
    from gunicorn.app.base import Application
    from gunicorn.six import iteritems

    # workers = multiprocessing.cpu_count() * 2 + 1
    workers = 1

    options = {
        'bind': '0.0.0.0:5000',
    }

    class GunicornRunner(Application):
        def __init__(self, app, options=None):
            self.options = options or {}
            self.application = app
            super(GunicornRunner, self).__init__()

        def load_config(self):
            config = dict([(key, value) for key, value in iteritems(self.options) if key in self.cfg.settings and value is not None])
            for key, value in iteritems(config):
                self.cfg.set(key.lower(), value)

        def load(self):
            return self.application

    GunicornRunner(app, options).run()
  1. Now if i run the server run_dev in debug mode db modifications are updated
  2. if run_server is used the modifications are not seen unless the app is restarted
  3. However if i run like gunicorn -c a.py app:app, the db updates are visible.

a.py contents

import multiprocessing

bind = "0.0.0.0:5000"
workers = multiprocessing.cpu_count() * 2 + 1

Any suggestions on where I am missing something..

Upvotes: 9

Views: 8926

Answers (3)

imbr
imbr

Reputation: 7632

Flask-Caching SimpleCache doesn't work w. workers > 1 Gunicorn

Had similar issue using version Flask 2.02 and Flask-Caching 1.10.1.

Everything works fine in development mode until you put on gunicorn with more than 1 worker. One probably reason is that on development there is only one process/worker so weirdly under this restrict circumstances SimpleCache works.

My code was:

app.config['CACHE_TYPE'] = 'SimpleCache' # a simple Python dictionary    
cache = Cache(app)

Solution to work with Flask-Caching use FileSystemCache, my code now:

app.config['CACHE_TYPE'] = 'FileSystemCache' 
app.config['CACHE_DIR'] = 'cache' # path to your server cache folder
app.config['CACHE_THRESHOLD'] = 100000 # number of 'files' before start auto-delete
cache = Cache(app)

Upvotes: 0

mobin
mobin

Reputation: 357

I also ran into this situation. Running flask in Gunicorn with several workers and the flask-cache won´t work anymore.

Since you are already using

app.config.from_object('default_config')  (or similar filename)

just add this to you config:

CACHE_TYPE = "filesystem"
CACHE_THRESHOLD = 1000000   (some number your harddrive can manage)
CACHE_DIR = "/full/path/to/dedicated/cache/directory/"

I bet you used "simplecache" before...

Upvotes: 3

TCB919
TCB919

Reputation: 123

I was/am seeing the same thing, Only when running gunicorn with flask. One workaround is to set Gunicorn max-requests to 1. However thats not a real solution if you have any kind of load due to the resource overhead of restarting the workers after each request. I got around this by having nginx serve the static content and then changing my flask app to render the template and write to static, then return a redirect to the static file.

Upvotes: 1

Related Questions