CODEWITHSUNDEEP
pythonhttpuploadpyramidwsgi
Riccardo Cagnasso
Riccardo Cagnasso

Reputation: 470

How to handle file upload asynchronously in pyramid

I'm writing a web application in python/pyramid that handles the upload of a file (via jquery uploader). The code for the upload is:

@view_config(route_name='file.upload', renderer='json')
def file_upload(request):

for item, filestorage in request.POST.items():
    f = File.create(filestorage)

    u = Upload.create(f.hash)

return {
    'url': request.route_url('file.get', uploadid=u.urlid)
}

where then File.create makes a sha1 of the file and moves it in a permanent location while putting the metadata in a database. (File is a SQLAlchemy class actually)

The problem here is that the view callable is called after that the file transfer to the server is complete. This poses two problems: the first one is that i'm unable to reject the file transfer if it's bigger than some size. The second is that i have to wait to receive the whole file and then i can start hashing it.

What i would like to obtain is to start process the file - some sort of stream - as soon i'm getting the data so i can hash it while the user is uploading it and i can stop the transfer if the size is bigger than some value.

Upvotes: 2

Views: 2082

Answers (2)

Sascha Gottfried
Sascha Gottfried

Reputation: 3329

In your case prefer client-side validation for file size limits. Using a jQuery plugin you require your clients to have Javascript enabled.

Look at pyramid application PyGall. It switched from using gp.fileupload to jQuery-File-Upload and injects file size limits from pyramid settings into jQuery plugin initialization. It computes md5 checksum as well and applies well to your use case.

Besides that there is a jQuery-File-Upload server-side implementation in pyramid. But from my point of view it is of limited use and is currently not running without some fixes.

Nevertheless a WSGI middleware like gp.fileupload plugged into your WSGI pipeline is able to do max_file_size checks.

Your second problem is about chunked file uploads. This increases complexity in your server-side upload handler implemented in pyramid, but enables you to create hashes from large files with less memory.

Upvotes: 3

Tiberiu Ichim
Tiberiu Ichim

Reputation: 671

Maybe you can find some help in this project:

https://pypi.python.org/pypi/gp.fileupload/

Upvotes: 0

Related Questions