CODEWITHSUNDEEP
ruby-on-railsrubyruby-on-rails-3amazon-web-servicesamazon-s3
notaceo
notaceo

Reputation: 1093

aws-s3 gem will connect in application.rb but fails on subsequent uses of API methods

My application.rb:

S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","s3_credentials.yml"))))["production"]
# AWS::S3::Base.establish_connection! S3_CREDENTIALS['connection']
AWS::S3::Base.establish_connection!(
:access_key_id     => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
:persistent        => true, # from http://www.ruby-forum.com/topic/110842

s3_credentials.yml:

 production: &defaults
  access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
  secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
  persistent: true

I launch my web server and in my customers_controller I check for the connection:

Rails.logger.info("S3 service connected? " + AWS::S3::Base.connected?.to_s)

And the log says it's connected:

S3 service connected? true

So I know the following:

My env variables are correct. My initializer and yml files are correct. Gem is fine

So then I do something simple like this in the controller:

@documents = Service.buckets()

I reload the page and here we go:

AWS::S3::InvalidAccessKeyId in CustomersController#edit

The AWS Access Key Id you provided does not exist in our records.

THE KICKER -

When I print my secret key and access key in plain text into the application.rb file (get rid of the s3_credentials.yml file, just put the keys into the appropriate lines in the application.rb file - I won't get the error.

Why is it that using environment variables with the aws-s3 gem fails spectacularly using the API methods but loads just fine? The keys can't be good and bad at the same time. It wouldn't connect at all if the keys were wrong, correct?

Upvotes: 1

Views: 1463

Answers (2)

notaceo
notaceo

Reputation: 1093

What I found after hours is that there is something inherently borked in the version of aws-s3 gem that I'm using. Period. There is no solution other than to fix the gem, and I don't have time to go that far. I checked the source and when to the connection.rb module to find where it's calling up the keys. It uses his own extract_keys! method, and after poking at that for a while I gave up.

I ended up using the aws-sdk gem. Config is a bit different than with aws-s3, but same ENV vars and it is all working now.

I submitted a request to the owner of the aws-s3 gem to take a look. But I don't know how active that gem is any longer.

aws-sdk is overkill when all you need is some simple access to S3 stuff, but whatever. It works.

Here is some of the config just in case someone finds it useful in the future:

aws.yml:

 development: &defaults
  access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
  secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
  persistent: true
  # bucket: YOUR BUCKET
  max_file_size: 10485760
  acl: public-read

NOTE : The <%= %> syntax above doesn't have spaces. Normally I format it with spaces. This might sound crazy to some, but if you put spaces between the tag and the value inside you will get an error from Amazon saying some crazy stuff about only requiring a single space. This drove me nuts. The leading or trailing space shouldn't matter! But it did. I took the spaces out - started working. Call me crazy. I spent hours battling this so you wouldn't have to!

application.rb (portion):

# Establish a base connection to Amazon S3
S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","aws.yml"))))["development"]
AWS.config(
:access_key_id     => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
)

And of course export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your environment.

On Mac just 'export AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE' and hit return.

Heroku is:

heroku config:set AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE

Can't do anything for you if you're using Windows.

Upvotes: 1

Jeremy Green
Jeremy Green

Reputation: 8574

You're right that it would never work if the key was just wrong. Have you tried to make sure that S3_CREDENTIALS['access_key_id'] and ENV['AWS_ACCESS_KEY_ID'] contain what you think they contain? My first guess is that S3_CREDENTIALS['access_key_id'] ends up containing the literal string of <%=ENV['AWS_ACCESS_KEY_ID']%> (not interpreted) since you're not processing the yaml file as erb before loading the file.

Upvotes: 0

Related Questions