CODEWITHSUNDEEP
rubyamazon-web-servicesamazon-s3carrierwavebuckets
schwabsauce
schwabsauce

Reputation: 447

copying s3 bucket to another aws account with CarrierWave Ruby gem

I'm trying to copy some contents from a bucket to a bucket in a different aws account. I begin by loading uploader objects into a hash. Then I attempted to connect to the other bucket and save the assets using the credentials for that bucket.

    task :product_color_images => :environment do
  CarrierWave.configure do |c|
    c.fog_credentials = {
      :provider               => 'AWS',
      :aws_access_key_id      => ENV['COPY_FROM_AWS_KEY_ID'],
      :aws_secret_access_key  => ENV['COPY_FROM_AWS_KEY']
    }

    c.fog_directory = 'orig-bucket' # bucket copied from
  end

  image_storage = {}

  ProductImage.all.each do |image|
    puts 'storing product image'
    image_storage[image.id] = image.image
  end

  CarrierWave.configure do |c|
    c.reset_config
    c.fog_credentials = {
      :provider               => 'AWS',
      :aws_access_key_id      => ENV['COPY_TO_AWS_KEY_ID'],
      :aws_secret_access_key  => ENV['COPY_TO_AWS_KEY']
    }

    c.fog_directory = 'target-bucket' # bucket copied to
  end

  image_storage.each do |k, v|
    image = ProductImage.find(k)
    image.image = v
    puts 'saving product image'
    image.save
  end
end

Trying to save a single image from one bucket to the other in a console reveals that the address of the target bucket is not used.

ruby-1.9.2-p290 :026 > image = ProductImage.find(197) 
ruby-1.9.2-p290 :027 > image.image = image_storage[197]
 => https://orig-bucket.s3.amazonaws.com/uploads/product_image/image/197/product_image.png 
ruby-1.9.2-p290 :028 > image.save
ruby-1.9.2-p290 :029 > image.image
 => https://orig-bucket.s3.amazonaws.com/uploads/product_image/image/197/product_image.png

Upvotes: 1

Views: 643

Answers (1)

Jeevan Dongre
Jeevan Dongre

Reputation: 4649

Sometimes it so happens that the bucket will be given enough permissions also make sure you have enough permission given to the images so that you can actually download them.

I have a better solution for you, what you can do is install and configure your s3cmd and do a rsync between two buckets. That will do things faster then your ruby on rails.

Upvotes: 1

Related Questions