CODEWITHSUNDEEP
pythonnginxgunicornpython-asyncioaiohttp
Claire T
Claire T

Reputation: 31

Deploying aiohttp.web Application using gunicorn and nginx

I am trying to deploy an aiohttp web app, but can't figure out how to get the app to serve over a unix socket, which I think I need in order to get nginx and gunicorn to talk to each other.

Simple example app from aiohttp documentation saved as app.py:

import asyncio
from aiohttp import web

@asyncio.coroutine
def hello(request):
    return web.Response(body=b'Hello')

app = web.Application()
app.router.add_route('GET', '/', hello)

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    handler = app.make_handler()
    f = loop.create_server(handler, '0.0.0.0', 8080)
    srv = loop.run_until_complete(f)
    try:
        loop.run_forever()
    except KeyboardInterrupt:
        pass
    finally:
        loop.run_until_complete(handler.finish_connections(1.0))
        srv.close()
        loop.run_until_complete(srv.wait_closed())
        loop.run_until_complete(app.finish())
    loop.close()

Running this with gunicorn directly works:

gunicorn -k aiohttp.worker.GunicornWebWorker -b 0.0.0.0:8000 app:app

But when I try to bind it instead to a unix socket, I get the following errors. 

gunicorn -k aiohttp.worker.GunicornWebWorker -b unix:my_sock.sock app:app

Traceback:

[2015-08-09 12:26:05 -0700] [26898] [INFO] Booting worker with pid: 26898
[2015-08-09 12:26:06 -0700] [26898] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/home/claire/absapp/venv/lib/python3.4/site-  packages/gunicorn/arbiter.py", line 507, in spawn_worker 
    worker.init_process()
  File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 28, in init_process
    super().init_process()
  File "/home/claire/absapp/venv/lib/python3.4/site-packages/gunicorn/workers/base.py", line 124, in init_process
    self.run()
  File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 34, in run
    self.loop.run_until_complete(self._runner)
  File "/usr/lib/python3.4/asyncio/base_events.py", line 268, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 236, in _step
    result = next(coro)
  File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 81, in _run
    handler = self.make_handler(self.wsgi, *sock.cfg_addr)
TypeError: make_handler() takes 4 positional arguments but 11 were given
[2015-08-09 12:26:06 -0700] [26898] [INFO] Worker exiting (pid: 26898)

I came across something in an aiohttp issue (https://github.com/KeepSafe/aiohttp/issues/136) that uses socket to create a socket to put as a parameter in the loop.create_server() function, but I just couldn't get anything to work. (I also don't know if the app in his code is the same web.Application object)

Does anybody know how I can make this work? Thanks!

Upvotes: 3

Views: 2894

Answers (1)

Yaroslav Admin
Yaroslav Admin

Reputation: 14535

The problem is that GunicornWebWorker doesn't support unix domain sockets. It comes from GunicornWebWorker.make_handler(self, app, host, port), which wants parameters: host and port. Obviously you don't have them if you're using unix socket, but have path to socket instead.

Let's take a look at the beginning of GunicornWebWorker._run():

def _run(self):
    for sock in self.sockets:
        handler = self.make_handler(self.wsgi, *sock.cfg_addr)
    ...

In case of -b localhost:8000 sock.cfg_addr is ['localhost', 8000], but for -b unix:my_sock.sock it's just 'my_sock.sock'. This is where error TypeError: make_handler() takes 4 positional arguments but 11 were given comes from. It unpacks string, instead of list.

The quick way to fix it is to subclass GunicornWebWorker and redefine GunicornWebWorker.make_handler() to ignore host and port. They are not used anyway. You can do it like this:

class FixedGunicornWebWorker(worker.GunicornWebWorker):
    def make_handler(self, app, *args):
        if hasattr(self.cfg, 'debug'):
            is_debug = self.cfg.debug
        else:
            is_debug = self.log.loglevel == logging.DEBUG

        return app.make_handler(
            logger=self.log,
            debug=is_debug,
            timeout=self.cfg.timeout,
            keep_alive=self.cfg.keepalive,
            access_log=self.log.access_log,
            access_log_format=self.cfg.access_log_format)

NOTE You'll need to have package with fixed worker in your PYTHONPATH. Otherwise Gunicorn won't be able to locate it. For example if you put fixed worker inside fixed_worker.py file inside the same directory you run gunicorn from, you can use it like:

$ PYTHONPATH="`pwd`:$PYTHONPATH" gunicorn -k fixed_worker.FixedGunicornWebWorker -b unix:my_sock.sock app:app

UPD Also opened issue in aiohttp repository.

Upvotes: 1

Related Questions