CODEWITHSUNDEEP
pythontornado
Zaar Hai
Zaar Hai

Reputation: 9889

How to catch write aborts in python-tornado?

I want to stream a long database result set through Tornado. I obviously need a server cursor since its not feasible to load the whole query in memory.

So I have the following code:

class QueryStreamer(RequestHandler):

    def get(self):
      cursor.execute("Select * from ...")
      chunk = cursor.fetch(1000)
      while chunk:
          self.write(chunk)
          self.flush()
          chunk = cursor.fetch(1000)        
     self.finish()
     cursor.close()

If someone does not read my request till the end? (i.e. curl ... |head), The get method keeps happily streaming my data to nowhere. I would expect to get SIGPIPE at some point and close database cursor (without running it to the end for nothing).

How can I catch write errors in Tornado?

Update: Following suggestion in the answer I've tried the following:

import tornado.ioloop
import tornado.web
import time

class PingHandler(tornado.web.RequestHandler):
        def get(self):
                for i in range(600):
                        self.write("pong\n")
                        self.flush()
                        time.sleep(1)
                        print "pong"
                self.finish()
                print "ponged"

        def on_connection_close(self):
                print "closed"

if __name__ == "__main__":
        application = tornado.web.Application([ ("/ping", PingHandler), ])
        application.listen(8888)
        tornado.ioloop.IOLoop.instance().start()

I'm running this file in terminal 1 and in terminal 2 I invoke:

curl -s   http://localhost:8888/ping

and after first response I hit CTRL-C. But in terminal 1 I see that it happily keeps "pong"-ing and on_connection_close never gets called.

Bottom line - still does not work.

Upvotes: 2

Views: 1021

Answers (2)

Jakub Roztocil
Jakub Roztocil

Reputation: 16252

You need to make the handler asynchronous and use ioloop.add_timeout instead of time.sleep, because that blocks the loop:

import tornado.ioloop
import tornado.web
import tornado.gen


class PingHandler(tornado.web.RequestHandler):

    connection_closed = False

    def on_connection_close(self):
        print "closed"
        self.connection_closed = True

    @tornado.gen.coroutine  # <= async handler
    def get(self):

        for i in range(600):

            if self.connection_closed:
                # `on_connection_close()` has been called,
                # break out of the loop
                break

            self.write("pong %s\n" % i)
            self.flush()

            # Add a timeout. Similar to time.sleep(1), but non-blocking:
            yield tornado.gen.Task(
                tornado.ioloop.IOLoop.instance().add_timeout,
                tornado.ioloop.IOLoop.instance().time() + 1,
            )

        self.finish()
        print "finished"

if __name__ == "__main__":
    application = tornado.web.Application([("/ping", PingHandler), ])
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

Upvotes: 7

daharon
daharon

Reputation: 2010

Implement the on_connection_close method and have it stop that write loop in your get handler.

Upvotes: 2

Related Questions