Jignesh Gohel
Jignesh Gohel

Reputation: 6552

Rails, Paperclip, Direct Upload to S3 - Authenticated URL returns SignatureDoesNotMatch response

rails (~> 3.2.13)

paperclip (2.7.4)

aws-sdk (1.8.0)

In my Rails application using Paperclip, I have a model named "Asset" which has an attachment named "upload" (s3_permissions: "private").

This attachment is directly uploaded to Amazon S3 first and then the Asset is saved.For doing the direct upload I have used following gem https://github.com/waynehoover/s3_direct_upload gem by following instructions at the tutorial http://www.blitztheory.com/direct-upload-with-s3_direct_upload/

Note:

The one thing which I ignored from the tutorial is that I am not using the class method copy_and_delete(paperclip_file_path, raw_source) to recreate the asset in bucket at a desired path.

I uploaded an image "Mars.gif" from my view to Amazon S3 first and associated it with the my Asset object.The image is successfully uploaded to the bucket and the Asset also got saved successfully.However after saving the Asset when I am viewing its details and trying to open the attached "Mars.gif" using the image's source url:

http://s3.amazonaws.com/mm_tom_test/uploads%2F1390421005820-yff5taw7kgf-da8b488d45d747deb206977d29e11005%2FMars.gif?AWSAccessKeyId=AKIAIMKHM4EATOHUAQ3Q&Expires=1390433963&Signature=ehSbrI2bKE4jqQNHyPJKWDySMyU%3D&response-content-disposition=inline&response-content-type=image%2Fgif

I am encountering SignatureDoesNotMatch error.Please find below the XML returned by Amazon S3:

    <?xml version="1.0" encoding="UTF-8"?>
    <Error>
       <Code>SignatureDoesNotMatch</Code>
       <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
       <StringToSignBytes>47 45 54 0a 0a 0a 31 33 39 30 34 33 33 39 36 33 0a 2f 6d 6d 5f 74 6f 6d 5f 74 65 73 74 2f 75 70 6c 6f 61 64 73 25 32 46 31 33 39 30 34 32 31 30 30 35 38 32 30 2d 79 66 66 35 74 61 77 37 6b 67 66 2d 64 61 38 62 34 38 38 64 34 35 64 37 34 37 64 65 62 32 30 36 39 37 37 64 32 39 65 31 31 30 30 35 25 32 46 4d 61 72 73 2e 67 69 66 3f 72 65 73 70 6f 6e 73 65 2d 63 6f 6e 74 65 6e 74 2d 64 69 73 70 6f 73 69 74 69 6f 6e 3d 69 6e 6c 69 6e 65 26 72 65 73 70 6f 6e 73 65 2d 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3d 69 6d 61 67 65 2f 67 69 66</StringToSignBytes>
       <RequestId>9E297018ADD4D5AD</RequestId>
       <HostId>EtMgiHpNfywzw7cNxAoCBFW5fY80LY3E5nTUuP182NfjzYqFTizIgjS+bgqPKM33</HostId>
       <SignatureProvided>ehSbrI2bKE4jqQNHyPJKWDySMyU=</SignatureProvided>
       <StringToSign>GET
    1390433963
    /mm_tom_test/uploads%2F1390421005820-yff5taw7kgf-da8b488d45d747deb206977d29e11005%2FMars.gif?response-content-disposition=inline&amp;response-content-type=image/gif</StringToSign>
       <AWSAccessKeyId>AKIAIMKHM4EATOHUAQ3Q</AWSAccessKeyId>
    </Error>

I had already spent enough hours to get this resolved with no success.I would really appreciate if anybody from the community can help me figure out what is causing this Signature Mismatch issue and how to get this resolved?

Listing below the code I have used:

/config/initializers/paperclip.rb

    Paperclip::ASSET_EXPIRATION_TIME = 3600.seconds

    Paperclip.interpolates(:key) do |attachment, style|
      attachment.instance.key
    end

    Paperclip.interpolates(:s3_conditional_url) do |attachment, style|
      attachment.expiring_url(Paperclip::ASSET_EXPIRATION_TIME, style)
    end

    module Paperclip::Storage::S3
      def public_url(style_name = default_style)
        if path
          "http://#{s3_host_name}/#{bucket_name}/#{path(style_name)}"
        end
      end

      def expiring_url(time = Paperclip::ASSET_EXPIRATION_TIME, style_name = default_style)
        if path
          s3_object(style_name).url_for(:read, :expires => time, :secure => use_secure_protocol?(style_name), :response_content_disposition => "inline",
        end
      end
    end

/config/amazon_s3.yml

development:
  access_key_id: 'AKIAIMKHM4EATOHUAQ3Q'
  secret_access_key: '<SECRET ACCESS KEY>'
  bucket: 'mm_tom_test'

/lib/s3_decider.rb

  module S3Decider
    def self.included(model)
      model.class_eval do
        if Paperclip::FILESYSTEM_ENVS.include?(Rails.env)
          @s3_decider = {
            :path => ":rails_root/public/assets/:class/:attachment/:id/:style/:basename.:extension",
            :url => "/assets/:class/:attachment/:id/:style/:basename.:extension"
          }
        else
          @s3_decider = {
            :storage        => :s3,
            :s3_credentials => "#{Rails.root}/config/amazon_s3.yml",
            :s3_permissions => "private",
            :s3_protocol    => Rails.env == "development" ? "http" : "https",
            :path           => ":class/:attachment/:id/:style.:extension",
            :url            => ":s3_conditional_url"
          }
        end
      end
    end
  end

/app/models/asset.rb

  class Asset < ActiveRecord::Base
    include S3Decider

    attr_accessible :upload, :upload_file_name, :upload_content_type, upload_file_size, :upload_updated_at

    paperclip_options_for_upload = @s3_decider.merge(
      path: ":key"
    )

    has_attached_file :upload, paperclip_options_for_upload

  end

Thanks.

Upvotes: 0

Views: 1642

Answers (1)

Jignesh Gohel
Jignesh Gohel

Reputation: 6552

I have resolved this issue myself and answering my own question in case anybody else ends up with the problem mentioned above.

S3DirectUpload gem in its uploadComplete callback receives the Amazon S3 bucket object's key as encoded-value and prefixed with a forward slash "/".For e.g. /uploads%2F1390554749261-nuflsns5cn-0de4e0f6e495e02bc5ee0c853d56b95f%2Fflower-3.jpeg

This same key if saved as it is in application's DB and then accessing the generated expiration URL using upload.expiring_url ( for an attachment named upload) shows SignatureDoesNotMatch returned from Amazon S3.

However if we sanitize the initially received key, before saving to application's database by:

  1. decoding the encoded values in the key
  2. removing the prefix forward slash ("/") from the key

then the generated expiration URL when accessed allows the resources to be accessed successfully.

In case anybody faces this issue I am giving here my working code snippet of the model.The other code snippets can be found in my above comments.

/app/models/asset.rb

  class Asset < ActiveRecord::Base
    include S3Decider

    attr_accessible :upload, :upload_file_name, :upload_content_type, upload_file_size, :upload_updated_at
    attr_accessible :key

    paperclip_options_for_upload = paperclip_options_for_upload.merge(
      path: ":key",
      url: ":s3_conditional_url",
      s3_url_options: lambda { |model|
        {
           response_content_type: model.upload_content_type,
           response_content_disposition: "inline"
        }
      }
    )

    has_attached_file :upload, paperclip_options_for_upload

    def key=(key)
      return if key.blank?
      # S3DirectUpload receives the Amazon S3 bucket object's key
      # as encoded and prefixed by a forward slash.For e.g.
      # /uploads%2F1390554749261-nuflsns5cn-0de4e0f6e495e02bc5ee0c853d56b95f%2Fflower-3.jpeg
      # Sanitizing it here else programmatically accessing the bucket object
      # corresponding to the key prefixed with "/" shall throw a No such key
      # exception.
      sanitized_key = key.sub(%r{^/},'')
      decoded_key = CGI.unescape(sanitized_key)
      write_attribute(:key, decoded_key)
    end

  end

Also while searching for solution I came across following pull request #769 which mentioned exactly the problem I was facing.

As I am using paperclip (2.7.4) its S3 storage don't contain the changes done in pull request.I verified by checking the source code in locally installed gem under rvm directory.Thus I changed the above mentioned code with following:

/config/initializers/paperclip.rb

Paperclip::ASSET_EXPIRATION_TIME = 3600.seconds

Paperclip.interpolates(:key) do |attachment, style|
  attachment.instance.key
end

Paperclip.interpolates(:s3_conditional_url) do |attachment, style|
  attachment.expiring_url(Paperclip::ASSET_EXPIRATION_TIME, style)
end

module Paperclip::Storage::S3
  def public_url(style_name = default_style)
    if path
      "http://#{s3_host_name}/#{bucket_name}/#{path(style_name)}"
    end
  end

  def expiring_url(time = Paperclip::ASSET_EXPIRATION_TIME, style_name = default_style)
     if path
     # Reference: https://github.com/thoughtbot/paperclip/pull/769
     base_options = { :expires => time, :secure => use_secure_protocol?(style_name) }
     s3_object(style_name).url_for(:read, base_options.merge(s3_url_options)).to_s
     end
  end

  # Reference: https://github.com/thoughtbot/paperclip/pull/769
  def s3_url_options
    s3_url_options = @options[:s3_url_options] || {}
    s3_url_options = s3_url_options.call(instance) if s3_url_options.is_a?(Proc)
    s3_url_options
  end
end

Thanks,

Jignesh

Upvotes: 2

Related Questions