CODEWITHSUNDEEP
python-3.xtornadopython-asyncio
m.awad
m.awad

Reputation: 187

run tornado with aiomysql in multi-process mode

I have been trying to run tornado with aiomysql in multi-process mode with the following code

@asyncio.coroutine
def get_mysql_connection(loop):
    return (yield from aiomysql.create_pool(host=host,port=3306,user=user, password=pass, db=db, loop=loop))


if __name__ == "__main__":
    tornado.platform.asyncio.AsyncIOMainLoop().install()
    ioloop = asyncio.get_event_loop()
    mysql = ioloop.run_until_complete(get_mysql_connection(ioloop))
    options.parse_config_file("app.conf")
    app = make_app(mysql)
    print('listening on %s:%s...' %(options.host, options.port))
    server = tornado.httpserver.HTTPServer(app)
    server.listen(options.port)
    server.start(0) #this is my problem
    ioloop.run_forever()

But I keep getting the below error

RuntimeError: Cannot run in multiple processes: IOLoop instance has already been initialized. You cannot call IOLoop.instance() before calling start_processes()

Everything is working fine except for the ioloop.start(0) line, is it possible to make the two libraries aiomysql and tornado work nicely in multi-process mode? If no what are my other options

Tornado version 4.4.2

python version 3.6.0

aiomysql version 0.0.9

Upvotes: 1

Views: 774

Answers (1)

Ben Darnell
Ben Darnell

Reputation: 22154

Like the message says, you can only fork worker processes if you do it before the IOLoop is initialized. This gets a little delicate with AsyncIOMainLoop since you want to install that as early as possible. The solution is to start your program with this sequence:

tornado.options.parse_config_file(...)
socks = tornado.netutil.bind_sockets(options.port, options.host)
tornado.process.fork_processes(0)
tornado.asyncio.AsyncIOMainLoop().install()
# Initialize the rest of your app, create the HTTPServer, 
# and instead of listen() or start(), do
server.add_sockets(socks)

what are my other options

In my opinion it's better to use an external process manager like supervisord instead of forking multiple processes within your app. That avoids most of these initialization-order traps.

Upvotes: 1

Related Questions