How to integrate web sockets with a django wsgi

Question

We have a significantly complex Django application currently served by apache/mod_wsgi and deployed on multiple AWS EC2 instances behind a AWS ELB load balancer. Client applications interact with the server using AJAX. They also periodically poll the server to retrieve notifications and updates to their state. We wish to remove the polling and replace it with "push", using web sockets.

Because arbitrary instances handle web socket requests from clients and hold onto those web sockets, and because we wish to push data to clients who may not be on the same instance that provides the source data for the push, we need a way to route data to the appropriate instance and then from that instance to the appropriate client web socket.

We realize that apache/mod_wsgi do not play well with web sockets and plan to replace these components with nginx/gunicorn and use the gevent-websocket worker. However, if one of several worker processes receive requests from clients to establish a web socket, and if the lifetime of worker processes is controlled by the main gunicorn process, it isn't clear how other worker processes, or in fact non-gunicorn processes can send data to these web sockets.

A specific case is this one: A user who issues a HTTP request is directed to one EC2 instance (host) and the desired behavior is that data is to be sent to another user who has a web socket open in a completely different instance. One can easily envision a system where a message broker (e.g. rabbitmq) running on each instance can be sent a message containing the data to be sent via web sockets to the client connected to that instance. But how can the handler of these messages access the web socket, which was received in a worker process of gunicorn? The high-level python web socket objects created gevent-websocket and made available to a worker cannot be pickled (they are instance methods with no support for pickling), so they cannot easily be shared by a worker process to some long-running, external process.

In fact, the root of this question comes down to how can web sockets which are initiated by HTTP requests from clients and handled by WSGI handlers in servers such as gunicorn be accessed by external processes? It doesn't seem right that gunicorn worker processes, which are intended to handle HTTP requests would spawn long-running threads to hang onto web sockets and support handling messages from other processes to send messages to the web sockets that have been attached through those worker processes.

Can anyone explain how web sockets and WSGI-based HTTP request handlers can possibly interplay in the environment I've described?

Thanks.

Answer 1

I think you've made the correct assesment that mod_wsgi + websockets is a nasty combination.

You would find all of your wsgi workers hogged by the web sockets and an attempt to (massively) increase the size of the worker pool would probably choke the server because of the memory usage and context switching.

If you like to stick with the synchronous wsgi worker architecture (as opposed to the reactive approach implemented by gevent, twisted, tornado etc), I would suggest looking into uWSGI as a application server. Recent versions can handle some URLs in the old way (i.e. your existing django views would still work the same as before), and route other urls to a async websocket handler. This might be a relatively smooth migration path for you.

Answer 2

It doesn't seem right that gunicorn worker processes, which are intended to handle HTTP requests would spawn long-running threads to hang onto web sockets and support handling messages from other processes to send messages to the web sockets that have been attached through those worker processes.

Why not? This is a long-running connection, after all. A long-running thread to take care of such a connection would seem... absolutely natural to me.

Often in these evented situations, writing is handled separately from reading.

A worker that is currently handling a websocket connection would wait for relevant message to come down from a messaging server, and then pass that down the websocket.

You can also use gevent's async-friendly Queues to handle in-code message passing, if you like.