CODEWITHSUNDEEP
phpcurluploadamazon-s3edgecast
krb
krb

Reputation: 16315

What is the best way to upload files to a storage service? (Amazon S3 or EdgeCast)

We have a web application which needs to store uploaded images with EdgeCast or Amazon S3. To do so we need to re-upload those images to EdgeCast / S3 first. There are a number of options I can think of:

  1. Do an inline curl re-upload, which will upload the image using the desired EdgeCast / S3 API. The disadvantage is that with concurrent uploads the server load will be massive.
  2. Queue re-uploads. I know it is possible but I have no idea how to do it.
  3. Don't bother with the re-uploads and upload directly to EdgeCast / S3 from the client end.
  4. Mount EdgeCast's / S3's FTP into the filesystem, and then just copy uploaded files, letting the FTP daemon do the rest.

Which is the best solution, and are there any other ones? I suspect it is either 1 or 2.

Edit: My application is in PHP

Upvotes: 1

Views: 2412

Answers (4)

Adrian Petrescu
Adrian Petrescu

Reputation: 17969

AWS provides an SDK for PHP to do this. It even has support for multi-part uploads, which is great news for developers.

It should be noted that the AWS SDK for PHP will also do retry logic and (if you're using Multipart, which you should) can resume failed uploads.

Upvotes: 0

craig
craig

Reputation: 11

EdgeCast storage supports Rsync/sFTP configuration to automatically sync content from a storage server. EdgeCast also supports "reverse-proxy" or "Customer Origin" configuration to pull content from another web-server into cache automatically. You can use S3 as this Customer-origin location or use EdgeCast's own Cloud Storage service or any other Web-server outside the EdgeCast network.

Upvotes: 1

fire
fire

Reputation: 21531

What I do is upload the files to a temp directory then have a cron script run that PUT's the files onto AWS, so as not to cause the upload process to take any longer for the end-user.

Upvotes: 0

Daveo
Daveo

Reputation: 19872

Not sure about EdgeCast but I know with Amazon S3 the best way to do this was to POST the file directly to the file server. See http://doc.s3.amazonaws.com/proposals/post.html

Doing it this way you would provide a HTML FORM with some field like folder id, file name, public key, timestamp etc to ensure it was secure and only you could upload to the server. The server would redirect their browser once the upload was complete from that page they redirected to you could inspect the query string to find out if the upload was successful or not then record the FileID into your DB.

Works great for reducing load on your server and gives the user a faster experience, however it can lead to orphan files if the upload succeeds but the DB insert fails.

Upvotes: 3

Related Questions