robb17
robb17

Reputation: 3

Flask-SocketIO 502 Error on AWS EC2 with [CRITICAL] Worker Timeouts

I'm setting up a reverse-proxy NGINX EC2 deployment of a flask app on AWS by following this guide. More specifically, I'm using a proxy pass to a gunicorn server (see config info below).

Things have been running smoothly, and the flask portion of the setup works great. The only thing is that, when attempting to access pages that rely on Flask-SocketIO, the client throws a 502 (Bad Gateway) and some 400 (Bad Request) errors. This happens after successfully talking a bit with the server, but then the next message(s) (e.g. https://example.com/socket.io/?EIO=3&transport=polling&t=1565901966787-3&sid=c4109ab0c4c74981b3fc0e3785fb6558) sit(s) at pending, and after 30 seconds the gunicorn worker throws a [CRITICAL] WORKER TIMEOUT error and reboots.

A potentially important detail: I'm using eventlet, and I've applied monkey patching.

I've tried changing around ports, using 0.0.0.0 instead of 127.0.0.1, and a few other minor alterations. I haven't been able to locate any resources online that deal with these exact issues.

The tasks asked of the server are very light, so I'm really not sure why it's hanging like this.

GNIX Config:

server {
    # listen on port 80 (http)
    listen 80;
    server_name _;
    location ~ /.well-known {
        root /home/ubuntu/letsencrypt;
    }
    location / {
        # redirect any requests to the same URL but on https
        return 301 https://$host$request_uri;
    }
}
server {
    # listen on port 443 (https)
    listen 443 ssl;
    server_name _;

    ...

    location / {
        # forward application requests to the gunicorn server
        proxy_pass http://127.0.0.1:5000;
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /socket.io {
        include proxy_params;
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass http://127.0.0.1:5000/socket.io;
    }

    ...
}

Launching the gunicorn server: gunicorn -b 127.0.0.1:5000 -w 1 "app:create_app()"

Client socket declaration:

var protocol =  window.location.protocol
var socket = io.connect(protocol + '//' + document.domain + ':' + location.port);

Requirements.txt:

Flask_SQLAlchemy==2.4.0
SQLAlchemy==1.3.4
Flask_Login==0.4.1
Flask_SocketIO==4.1.0
eventlet==0.25.0
Flask==1.0.3
Flask_Migrate==2.5.2

Sample client-side error messages:

POST https://example.com/socket.io/?EIO=3&transport=polling&t=1565902131372-4&sid=17c5c83a59e04ee58fe893cd598f6df1 400 (BAD REQUEST)

socket.io.min.js:1 GET https://example.com/socket.io/?EIO=3&transport=polling&t=1565902131270-3&sid=17c5c83a59e04ee58fe893cd598f6df1 400 (BAD REQUEST)

socket.io.min.js:1 GET https://example.com/socket.io/?EIO=3&transport=polling&t=1565902165300-7&sid=4d64d2cfc94f43b1bf6d47ea53f9d7bd 502 (Bad Gateway)

socket.io.min.js:2 WebSocket connection to 'wss://example.com/socket.io/?EIO=3&transport=websocket&sid=4d64d2cfc94f43b1bf6d47ea53f9d7bd' failed: WebSocket is closed before the connection is established

Sample gunicorn error messages (note: first line is the result of a print statement)

Client has joined his/her room successfully
[2019-08-15 20:54:18 +0000] [7195] [CRITICAL] WORKER TIMEOUT (pid:7298)
[2019-08-15 20:54:18 +0000] [7298] [INFO] Worker exiting (pid: 7298)
[2019-08-15 20:54:19 +0000] [7300] [INFO] Booting worker with pid: 7300

Upvotes: 0

Views: 910

Answers (1)

Miguel Grinberg
Miguel Grinberg

Reputation: 67479

You need to use the eventlet worker with Gunicorn:

gunicorn -b 127.0.0.1:5000 -w 1 -k eventlet "app:create_app()"

Upvotes: 0

Related Questions