CODEWITHSUNDEEP
file-uploadtornado
PLSD
PLSD

Reputation: 21

uploading large files in tornado using @stream_request_body

@stream_request_body class StreamHandler(RequestHandler): def post(self): self.temp_file.close()

def prepare(self):

     max_buffer_size = 4 * 1024**3 # 4GB
     self.request.connection.set_max_body_size(max_buffer_size)
     self.temp_file = open("test.txt","w")


def data_received(self, chunk):
    self.temp_file.write(chunk)

With the above code I am able to upload file but in raw data form as shown below

-----------------------------6552719992117258671800152707 Content-Disposition: form-data; name="dest"

csv -----------------------------6552719992117258671800152707 Content-Disposition: form-data; name="carrier"

bandwidth -----------------------------6552719992117258671800152707 Content-Disposition: form-data; name="file1"; filename="test.csv" Content-Type: text/csv

And the content of uploaded file follows here.

How do I get the request parameters parsed from the file and separate the data of the file uploaded? Is there any other way to upload large files(around 2 GB) in tornado

Upvotes: 0

Views: 1265

Answers (1)

Ben Darnell
Ben Darnell

Reputation: 22134

This is the multipart protocol used by HTML forms. Tornado can currently only parse this if it sees all the data at once, not in a streaming upload. There's a third-party library that should be able to handle this: https://github.com/siddhantgoel/streaming-form-data. See this issue for more: https://github.com/tornadoweb/tornado/issues/1842

If you control the client, it may be simpler to use a plain HTTP PUT instead of the HTML multipart form protocol. This doesn't require any special handling on the server side

Upvotes: 1

Related Questions