CODEWITHSUNDEEP
rubyencodingamazon-s3
twooface
twooface

Reputation: 450

Can't read files from amazon s3 bucket using aws_s3 (ruby gem) in correct encoding?

I have problem when creating a file in encoding 'utf-8' and reading it from amazon-s3 bucket.

I create a file.

file = File.open('new_file', 'w', :encoding => 'utf-8')
string = "Some ££££ sings"
file.write(string)
file.close

When read from local everything is ok.

open('new_file').read
=> "Some ££££ sings"

Now I upload the file to amazon s3 using aws_s3.

AWS::S3::S3Object.store('new_file', open('new_file'), 'my_bucket')
=> #<AWS::S3::S3Object::Response:0x2214462560 200 OK>

When I read from amazon s3

AWS::S3::S3Object.find('new_file', 'my_bucket').value
=> "Some \xC2\xA3\xC2\xA3\xC2\xA3\xC2\xA3 sings"

open(AWS::S3::S3Object.find('new_file','my_bucket').url).read
=> "Some \xC2\xA3\xC2\xA3\xC2\xA3\xC2\xA3 sings"

I've been trying many things a still can't find solution.

Many Thanks for all the help

M

Upvotes: 5

Views: 2679

Answers (2)

Overbryd
Overbryd

Reputation: 5006

I think there is a better solution. Put the file you are writing to in binmode.

file = File.open("test.txt", "wb")

# or use File#binmode
file = File.open("test.txt")
file.binmode

# binmode also works with Tempfile
file = Tempfile.new
file.binmode

# then proceed to downloading
s3 = AWS::S3.new
s3.buckets["foo"]["test.txt"].read do |chunk|
  file.write(chunk)
end

Upvotes: 0

twooface
twooface

Reputation: 450

I found solution on different forum.

They way to do it is to make sure we are passing/uploading the text file in 'utf-8' in the first place. This it self will not solve the problem but will allow you with certainty force on stream back string encoding.

open(AWS::S3::S3Object.find('new_file','my_bucket').url).read.force_encoding('utf-8')

Upvotes: 4

Related Questions