Large file download for a Rails project

Question

One client project will be online two months later. One of the requirements changed is to support large files (10 to 15MB per RAW camera file, expected 1000 to 5000 files download per day) download worldwide for their customers. The process will be:

• there is upload screen via paperclip to the rails local public folder
• a hourly task to upload to web storage (S3?)
• update the download url from paperclip url to the web url

Questions:

1. is there a gem/plug-in for this purpose?
2. if no, any gem/plug-in for S3 to recommend?

Questions about the storage provider:

1. is S3 recommended?
2. or other service to recommend?

The baseline is: the client's web server does not and will not have the bandwidth to handle the downloads.

Thanks

Answer 1

If you end up going the route of uploading directly to S3 which offloads the work from your Rails server and makes it asynchronous, please check out my sample projects:

Sample project using Rails 3, Flash and MooTools-based FancyUploader to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-FancyUploader

Sample project using Rails 3, Flash/Silverlight/GoogleGears/BrowserPlus and jQuery-based Plupload to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-Plupload

Answer 2

I don't think there is anything that will do all of this out of the box for you. Paperclip will push files sychronousy to S3 on upload, so you will need to make this ansychronous yourself.

S3 is rock-solid, I have used it in production on a number of projects. Totally recommended.

You can upload files directly to S3 which may help by reducing the double handling of the file (no longer need to upload to your app before pushing to Amazon):

http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=139&externalID=1434

Answer 3

The aws-s3 and delayed_job gems are probably what you want.

gem install aws-s3

S3 is popular and widely used as far as I am aware.