CODEWITHSUNDEEP
rubyamazon-s3amazon-web-services
Philip Wiebe
Philip Wiebe

Reputation: 157

Ruby AWS::S3::S3Object (aws-sdk): Is there a method for streaming data as there is with aws-s3?

In aws-s3, there is a method (AWS::S3::S3Object.stream) that lets you stream a file on S3 to a local file. I have not been able to locate a similar method in aws-sdk.

i.e. in aws-s3, I do:

File.open(to_file, "wb") do |file|
  AWS::S3::S3Object.stream(key, region) do |chunk|
    file.write chunk
  end
end

The AWS::S3:S3Object.read method does take a block as a parameter, but doesn't seem to do anything with it.

Upvotes: 6

Views: 3431

Answers (2)

Trevor Rowe
Trevor Rowe

Reputation: 6528

The aws-sdk gem now supports chunked reads of objects in S3. The following example gives a demonstation:

s3 = AWS::S3.new
File.open(to_file, "wb") do |file|
  s3.buckets['bucket-name'].objects['key'].read do |chunk|
    file.write chunk
  end
end

Upvotes: 6

awendt
awendt

Reputation: 13723

At this time, not officially. I found this thread in the official AWS Ruby forum:

Does the ruby aws gem support streaming download from S3. Quoting AWS staff:

Unfortunately there is not a good solution baked into the aws-sdk gem. We are looking into way we could make this much simpler for the end user.

There's sample code for downloading in chunks. You might want to have a look at that for inspiration.

Upvotes: 2

Related Questions