CODEWITHSUNDEEP
pythontornadopython-requests
erratbi
erratbi

Reputation: 97

Stream a remote file with tornado and requests

So basicaly what i'm trying to do is to open connection to a remote file and read it in chunks then spell it the user

using tornado as a web framework and requests for reading the remote file

here is where i got so far

class DefaultHandler(tornado.web.RequestHandler):
def get(self):
    url = 'http://domain.tld/large_file.rar'
    r = requests.get(url, stream=True)
    self.set_header('Content-Type', 'application/force-download')
    self.set_header('Content-Disposition', 'attachment; filename=large_file.rar')
    self.set_header('Content-length', r.headers.get('content-length'))
    self.set_header('Pragma', 'public')
    self.set_header('Expires', 0)
    self.set_header('Cache-Control', 'must-revalidate, post-check=0, pre-check=0')
    self.set_header('Cache-Controle', 'public')
    self.set_header('Content-Transfer-Encoding', 'binary')
    self.set_header('Accept-Ranges', 'bytes')
    self.set_header('Connection', 'close')
    for chunk in r.iter_content(chunk_size=1024):
        if chunk:
            self.write(chunk)
            self.flush()
    self.finish()

EDIT------------------------------------------

class DefaultHandler(tornado.web.RequestHandler):
def get(self):

    url = 'http://largefilelocation'
    r = requests.Request('GET', url).prepare()
    resp = requests.Session().send(r, stream=True)

    self.set_header('Content-Type', 'application/force-download')
    self.set_header('Content-Disposition', resp.headers['content-disposition'])
    self.set_header('Pragma', 'public')
    self.set_header('Expires', 0)
    self.set_header('Content-Length', int(resp.headers['content-length']))
    self.set_header('Cache-Control', 'must-revalidate, post-check=0, pre-check=0')
    self.set_header('Cache-Controle', 'public')
    self.set_header('Content-Transfer-Encoding', 'binary')
    self.set_header('Accept-Ranges', 'bytes')
    self.set_header('Connection', 'close')

    for chunk in self._file_stream(resp):
        self.write(chunk)
        self.flush()
    self.finish()

def _file_stream(self, resp):
    print resp.headers
    for chunk in resp.iter_lines(128):
        if not chunk: break
        yield chunk

Upvotes: 0

Views: 1196

Answers (1)

Ben Darnell
Ben Darnell

Reputation: 22134

The requests library is synchronous, which means that it blocks the IOLoop whenever you use it. You have to use Tornado's AsyncHTTPClient instead to allow the IOLoop to keep running. AsyncHTTPClient has a streaming_callback option that you can use here.

Upvotes: 3

Related Questions