Tornado web asynchronous

Question

I am trying to play with this piece of code to understand @tornado.web.asynchronous. The code as intended should handle asynchronous web requests but it doesnt seem to work as intended. There are two end points:

1) http://localhost:5000/A  (This is the time consuming request and 
takes a few seconds)
2) http://localhost:5000/B (This is the fast request and takes no time to return.

However when I hit the browser to go to http://localhost:5000/A and then while that is running go to http://localhost:5000/B the second request is queued and runs only after A has finished. In other words one task is time consuming but it blocks the other faster task. What am I doing wrong?

import tornado.web
from tornado.ioloop import IOLoop

import sys, random, signal


class TestHandler(tornado.web.RequestHandler):

    """
    In below function goes your time consuming task
    """


   def background_task(self):
      sm = 0
      for i in range(10 ** 8):
         sm = sm + 1
      return str(sm + random.randint(0, sm)) + "\n"

   @tornado.web.asynchronous
   def get(self):
      """ Request that asynchronously calls background task. """
      res = self.background_task()
      self.write(str(res))
      self.finish()


class TestHandler2(tornado.web.RequestHandler):

   @tornado.web.asynchronous
   def get(self):
      self.write('Response from server: ' + str(random.randint(0, 100000)) + "\n")
      self.finish()

   def sigterm_handler(signal, frame):
     # save the state here or do whatever you want
     print('SIGTERM: got kill, exiting')
     sys.exit(0)

   def main(argv):
     signal.signal(signal.SIGTERM, sigterm_handler)
     try:
       if argv:
         print ":argv:", argv
         application = tornado.web.Application([
        (r"/A", TestHandler),
        (r"/B", TestHandler2),
        ])

        application.listen(5000)
        IOLoop.instance().start()
      except KeyboardInterrupt:
        print "Caught interrupt"
     except Exception as e:
        print e.message
     finally:
        print "App: exited"

if __name__ == '__main__':
   sys.exit(main(sys.argv))

Answer 1

According to the documentation:

To minimize the cost of concurrent connections, Tornado uses a single-threaded event loop. This means that all application code should aim to be asynchronous and non-blocking because only one operation can be active at a time.

To achieve this goal you need to prepare the RequestHandler properly. Simply adding @tornado.web.asynchronous decorator to any of the functions (get, post, etc.) is not enough if the function performs only synchronous actions.

What does the @tornado.web.asynchronous decorator do?

Let's look at the get function. The statements are executed one after another in a synchronous manner. Once the work is done and the function returns the request is being closed. A call to self.finish() is being made under the hood. However, when we use the @tornado.web.asynchronous decorator the request is not being closed after the function returned. So the self.finish() must be called by the user to finish the HTTP request. Without this decorator the request is automatically finished when the get() method returns.

Look at the "Example 21" from this page - tornado.web.asynchronous:

@web.asynchronous
def get(self):
  http = httpclient.AsyncHTTPClient()
  http.fetch("http://example.com/", self._on_download)

def _on_download(self, response):
  self.finish()

The get() function performs an asynchronous call to the http://example.com/ page. Let's assume this call is a long action. So the http.fetch() function is being called and a moment later the get() function returns (http.fetch() is still running the background). The Tornado's IOLoop can move forward to serve the next request while the data from the http://example.com/ is being fetched. Once the the http.fetch() function call is finished the callback function - self._on_download - is called. Then self.finish() is called and the request is finally closed. This is the moment when the user can see the result in the browser.

It's possible due to the httpclient.AsyncHTTPClient(). If you use a synchronous version of the httpclient.HTTPClient() you will need to wait for the call to http://example.com/ to finish. Then the get() function will return and the next request will be processed.

To sum up, you use @tornado.web.asynchronous decorator if you use asynchronous code inside the RequestHandler which is advised. Otherwise it doesn't make much difference to the performance.

EDIT: To solve your problem you can run your time-consuming function in a separate thread. Here's a simple example of your TestHandler class:

class TestHandler(tornado.web.RequestHandler):
    def on_finish(self, response):
        self.write(response)
        self.finish()

    def async_function(base_function):
        @functools.wraps(base_function)
        def run_in_a_thread(*args, **kwargs):
            func_t = threading.Thread(target=base_function, args=args, kwargs=kwargs)
            func_t.start()
        return run_in_a_thread

    @async_function
    def background_task(self, callback):
        sm = 0
        for i in range(10 ** 8):
            sm = sm + 1
        callback(str(sm + random.randint(0, sm)))

    @tornado.web.asynchronous
    def get(self):
        res = self.background_task(self.on_finish)

You also need to add those imports to your code:

import threading
import functools
import threading

async_function is a decorator function. If you're not familiar with the topic I suggest to read (e.g.: decorators) and try it on your own. In general, our decorator allows the function to return immediately (so the main program execution can go forward) and the processing to take place at the same time in a separate thread. Once the function in a thread is finished we call a callback function which writes out the results to the end user and closes the connection.