CarrierWave_Direct gem: Keep current directory structure on S3

Question

so I have a site already in production using the carrierwave gem with images stored on amazon s3. my uploader uses the store_dir method to specify a particular structure to put in processed images.

well, now in my dev environment I've added the carrierwave_direct gem to start uploading directly to S3. The problem is this gem completely overrides the store_dir and filename defaults in my uploader. I can't push my fully working uploader live because all my old image links will be broken.

it's my understanding that the CWdirect gem would upload a raw image file to a "temp" directory on S3, then S3 responds and gives you a key variable so you can grab this file and process it as you see fit. so, should i be using a completely separate image uploader class in carrierwave to process the images and place them in the correct folders? meaning I'll have one uploader dedicated to carrierwave_direct that uploads wherever this gem seems to want to upload to; and I'll use another uploader.rb class linked to my real model that keeps my current store_dir and filename structure?

In any case, my basic question is, how can I use CarrierWave_Direct gem if I already have CW running in production with images in a specific folder structure?

Answer 1

Ok, so I did figure out how to do this and I'll explain below. My hunch was correct to use two different CarrierWave uploader classes--one class dedicated to uploading to S3 (using CarrierWave_Direct gem), and a second class used only for image processing (the class I was already using in production). I'll try and post relevant code below but if anyone has questions let me know. I'm unsure why I haven't seen others using separate classes like this but it seems to work for me.

My image Uploader class app\uploaders\image_uploader.rb utilizing carrierwave_direct gem:

class ImageUploader < CarrierWave::Uploader::Base
  include CarrierWaveDirect::Uploader

  include ActiveModel::Conversion
  extend ActiveModel::Naming

  include CarrierWave::MimeTypes
  process :set_content_type

  # Add a white list of extensions which are allowed to be uploaded.
  # For images you might use something like this:
  def extension_white_list
    %w(jpg jpeg gif png)
  end

  # Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
  include Sprockets::Helpers::RailsHelper
  include Sprockets::Helpers::IsolatedHelper

  # Override the directory where uploaded files will be stored.
  # CarrierWaveDirect::Uploader puts raw uploaded files in this directory on S3 as a first step
  def store_dir
    "unprocessed_uploads"
  end
end

**notice there's no processing being done in this class

My image PROCESSING class app\uploaders\image_processor.rb (what was already in place in production):

class ImageProcessor < CarrierWave::Uploader::Base

  # Include RMagick or MiniMagick support:
  # include CarrierWave::RMagick
  include CarrierWave::MiniMagick
  include CarrierWave::MimeTypes
  process :set_content_type

  # Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
  include Sprockets::Helpers::RailsHelper
  include Sprockets::Helpers::IsolatedHelper

  # Choose what kind of storage to use for this uploader:
  storage :fog

  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
    # "uploads/#{model.class.to_s.underscore}/path/#{model.id}"
  end

  # Provide a default URL as a default if there hasn't been a file uploaded:
  def default_url
    "logos/" + [version_name, "default.png"].compact.join('_')
  end

  # Process files fetched from S3 after they are uploaded:

  def make_thumbnail(width, height)
    # uses MiniMagick classes to get a square, centered thumbnail image
    manipulate! do |img|
      if img[:width] < img[:height]
        remove = ((img[:height] - img[:width])/2).round
        img.shave("0x#{remove}")
      elsif img[:width] > img[:height]
        remove = ((img[:width] - img[:height])/2).round
        img.shave("#{remove}x0")
      end

      img.resize("#{width}x#{height}")
      img
    end        
  end


  # Create different versions of your uploaded files:
  # the process statement below isn't defined within a version block on purpose--this means the ORIGINAL uploaded photo is constrained to 1050 pics
  process :resize_to_limit => [1050, 1050]
  process :quality => 85 # this reduces filesize greatly and saves space

  version :thumb do
    process :make_thumbnail => [100, 100]
    process :quality => 85 # this reduces filesize greatly and saves space
  end

  version :big_thumb do
    process :make_thumbnail => [350, 350]
    process :quality => 85 # this reduces filesize greatly and saves space
  end

  # Add a white list of extensions which are allowed to be uploaded.
  # For images you might use something like this:
  def extension_white_list
    %w(jpg jpeg gif png)
  end

  # Override the filename of the uploaded files:
  # Avoid using model.id or version_name here, see uploader/store.rb for details.
  def filename
    if original_filename
      if model && model.read_attribute(:image).present?
        model.read_attribute(:image)
      else
        "#{secure_token}.#{file.extension}"
      end
    end
  end

protected
  def secure_token
    var = :"@#{mounted_as}_secure_token"
    model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid)
  end

end

My Photo model (summarized):

class Photo < ActiveRecord::Base
  mount_uploader :image, ImageProcessor

  def save_and_process_image(options = {})
    s3_unprocessed_image_url = self.image.asset_host + '/' + self.key
      # this next line downloads the image from S3
      # and this save line below will process the image and reupload to S3 according to ImageProcessor settings
      self.remote_image_url = s3_unprocessed_image_url
      save
  end

end

I also have Photo controller and view code available, if ya'll want it let me know. Basically I use the ImageUploader class to do the initial upload to S3 to a folder called unprocessed_uploads. Then S3 responds with a key field in the URL which I pass on to ImageProcessor class--this is attached to a Photo and processes the thumbnail and other images then re-uploads them to my uploads folder on S3.

This separation meant i didn't need to change my current folder structure on S3 when adding the carrierwave_direct gem. Hope this helps others. Let me know if you need more code I'm kind of tired of typing :)

UPDATE--Adding more code:

Photos Controller:

class PhotosController < ApplicationController

  def index
    @photos = @photos.sort_by(&:created_at)

    @uploader = ImageUploader.new
    @uploader.success_action_redirect = new_tank_photo_url(@tank)

    respond_to do |format|
      format.html # index.html.erb
      format.json { render json: @photos }
    end
  end

  def create
    respond_to do |format|

      if @photo.save_and_process_image
        format.html { redirect_to tank_photos_path(@tank), notice: 'Photo uploaded successfully and is being processed...' }
        format.json { render json: @photo, status: :created, location: @photo }
      else
        format.html { render :new }
        format.json { render json: @photo.errors, status: :unprocessable_entity }
      end
    end
  end

Photos Index view, the form with the Upload button:

    <%= direct_upload_form_for @uploader, :html => {:class => "form-inline"} do |f| %>
        <%= f.file_field :image %>
        <%= f.submit "Upload", :class => "btn btn-primary btn-medium" %>
    <% end %>

So with the above view/controller code added, here is a summary of the steps taken. Please note the difference between the ImageUploader class and the ImageProcessor class:

1. In my Photos Controller I create the @uploader variable which is of class ImageUploader--this means that images submitted in my index/view form which uses @uploader will upload the image to a "temp" folder--Since this uses the ImageUploader class then no processing is done when this image is uploaded (yet).
2. When the user clicks "Upload" button on the form, the Photo>>create action is called. This action calls @photo.save_and_process_image -- look at the model to see that this method actually grabs the recently uploaded image from the S3 'temp' folder, processes it, then re-uploads the processed images to their final destination. This is all possible because my Photo model is linked to the ImageProcessor class, which does the processing/uploading. It's not linked to the separate ImageUploader class.

Hope this helps explain what i have going on