CODEWITHSUNDEEP
c#.netasp.net-mvcamazon-s3
Mark Sowul
Mark Sowul

Reputation: 10600

Reading stream from AWS S3 involves loading entire file

The short version of my scenario is that I am reading a file stored in Amazon's S3 with the .NET SDK...

GetObjectRequest request = new GetObjectRequest
{
    BucketName = this.m_bucketName,
    Key = GetFileKey(fileIdentifier),
};

IAmazonS3 source = ...
GetObjectResponse response = await source.GetObjectAsync(request);
return response.ResponseStream;

I then pass this stream through to MVC as a File result

public async Task<FileResult> Download(...)
{
   return File(GetAwsStream(...), ...);
}

The problem is, apparently S3 is eagerly calculating a checksum of the entire file before returning anything. For large files, this is a significant issue because

This entirely defeats the point of a stream. Is there any way to get an actual "stream" from S3?

Upvotes: 4

Views: 3543

Answers (1)

jzonthemtn
jzonthemtn

Reputation: 3404

You can use the HTTP Range header to, in a loop, download specific bytes of an S3 object and then pass those bytes to the client once they are downloaded. That way the web server won't have to wait until the full file has been retrieved to give the client something.

Upvotes: 3

Related Questions