Multiple workers with gevent-socketio fails with xhr-polling transport because of sessions

Question

I'm currently trying to scale gevent-socketio across multiple workers with the gunicorn server using the worker socketio.sgunicorn.GeventSocketIOWorker. I'm using websockets when it exists otherwise I'm forcing XHR-polling (for IE etc).

XHR-polling needs a session to keep track of the following polls but as soon as I go from one to two or more workers the requests starts spreading out between themselves which means the state is lost and everything breaks down.

I think the following lines of code is relevant: https://github.com/abourget/gevent-socketio/blob/master/socketio/handler.py#L104-106 I suppose I need some other storage engine, for example redis which I'm using for regular pubsub-actions but this is deep inside the actual library.

So my question is how do I go from in-memory session storage to another backend engine globally in my application (does it gracefully override the session code in the link above?) without having to modify the library itself? Something like PHP's session directives in php.ini. I suppose an argument could be made that this is a very generic python question but I'm having trouble to find relevant information, and I'm also unsure that it'll work for this library.

Or in the alternative, how do I use gevent-socketio's xhr-polling transport across different workers and servers (without stickyness)?

Thanks!

Answer 1

Just a thought, as I am having the same issue; instead of letting gunicorn fork (with the -w flag), maybe you could spawn multiple process of gunicorn on different ports, then use nginx to balance on them using an "upstream" block with "sticky" session. I believe that is how the nodejs implementation handles multiprocessing of workers when not sharing states with Redis.

Answer 2

Look at the examples here:

https://github.com/abourget/gevent-socketio/tree/master/examples/pyramid_backbone_redis_chat_persistence

It shows how to use redis as a distributed message queue

Answer 3

This is obviously a limitation of socketio. From what I can see on the web, session handling is typically done at web framework layer rather than web server one. socketio tries to do it on its own, lower layer, and does it in a limited manner. I guess the authors thought a full-fledged solution would be an overkill. In your case, they proved to be wrong.

There are only two ways to overcome limitations that require logic change: patching the source and patching in the runtime. Choose whichever pleases you the most (or, well, disgusts the least :^) ). For the 2nd option, I suggest to replace request_tokens and/or the code that creates it with another entity with the same interface. Due to the reasons stated in the 1st paragraph, I really think that socketio authors are likely to accept a source patch that would allow it to utilize external session handling mechanisms if you propose one.

Standard locations for session information are: shared memory, files, database. I suggest you change the logic in such a way that socketio uses the same mechanism as your web framework (or whatever composes your pages) does.