CODEWITHSUNDEEP
asp.net-mvcamazon-s3
Adrian
Adrian

Reputation: 1147

Upload large files to s3 with ASP.net MVC

There are plenty of similar questions on Stack Overflow, but I'm having trouble uploading a large file to S3 via MVC.net.

Based on the aws .net sdk examples, and some other answers, I have the following:

public static void UploadToS3(System.IO.Stream stream, string bucketName, string keyName, bool asAttachement)
    {
        AmazonS3 client;
        string accessKey = Settings.ApplicationSettings.AccessKeyID;
        string secretKey = Settings.ApplicationSettings.SecretAccessKeyID;
        // This config ensures s3 works in medium trust.
        AmazonS3Config s3config = new AmazonS3Config();
        s3config.UseSecureStringForAwsSecretKey = false;
        using (client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey, s3config))
        {
            try
            {
                PutObjectRequest request = new PutObjectRequest();
                request.WithBucketName(bucketName)
                    .WithKey(keyName)
                    .WithInputStream(stream);


                if (asAttachement)
                {
                    // add headers to ensure asset is downloaded, not opened in browser.
                    request.AddHeaders(Amazon.S3.Util.AmazonS3Util.CreateHeaderEntry("Content-Disposition", "attachment"));
                }

                S3Response response = client.PutObject(request);
                response.Dispose();
            }
            catch(Exception ex)
            {
                throw new ApplicationException("Unable to upload asset." + ex.Message);
            }
        }
    }

The web app works perfectly with small files, and large files will upload to the site (hosted with Rackspace Cloud Sites), but the browser times out after about 30 seconds past the http post, which is not long enough to upload the stream to Amazon. The browser times out as not receiving a response from the server.

What do I need to do to make this work for large files, or how should I be uploading a file to S3?

Upvotes: 0

Views: 2280

Answers (1)

derekdb
derekdb

Reputation: 96

From the sound of it, you have 3 systems involved in this. There is a client running a web-browser, there is your server, and there is S3. As I understand your scenario, the client makes a request against your server, causing your server to upload a file to S3. You mention that the browser is complaining that the server is not responding. That is probably because your server is busy uploading the file to S3. While your server is uploading the object to S3, it is not sending any status updates to the browser. If the upload takes long enough, the browser will give up waiting. That doesn't mean that the upload didn't happen, just that the browser stopped waiting for your server to respond. What you need to do is return some kind of incremental result to the client. I'm not a .Net programmer, but what I would probably do is to spawn an new thread to do the upload, and on the request thread return some initial content indicating that the request was underway, flush that response but not close the output. When have a loop that waited for the upload thread, waking up every 5 seconds to writing a space character and flushing that to the client. That will keep the browser from timing-out.

Upvotes: 1

Related Questions