CODEWITHSUNDEEP
ruby-on-rails
Joy
Joy

Reputation: 4483

Saving and later importing xls, csv files in rails application

In my ruby on rails application, have implemented a facility seeing the railscast video http://railscasts.com/episodes/396-importing-csv-and-excel, that will directly import and update the db at same time, but it is creating problems when the file is too large.

So I want to write a facility to upload a csv or excel file in my application and save it in a directory. Then I want to add some kind of observer that will observe the contents of the directory and on an event like create or update of a file in that directory will trigger contents of those files to be uploaded in db. I am getting no idea about how to approach this. Thanks in advance.

Upvotes: 1

Views: 992

Answers (2)

Max Williams
Max Williams

Reputation: 32943

As an alternative to Resque (and i do think doing it in a background job is the best approach), see also Spawnling, previously known as Spawn:

https://github.com/tra/spawnling

Super-low maintenance. In your controller action do something like

@file = <uploaded file here>
spawn do 
  #perform some long-running process using @file, in a new process
end
#current thread carries on straight away.

If you need to test if it's finished (or crashed for that matter) you can save the id of the new process like so:

@file = <uploaded file here>
spawner = spawn do 
  #perform some long-running process using @file, in a new process
end

#current thread carries on straight away.

spawner = spawn do
  #make it in a temp file so serve_if_present doesn't serve a halfmade file
  FileUtils.rm @filename if File.exists?(@filename)
  temp_filename = "#{@filename}.temp"
  ldb "temp_filename = #{temp_filename}"
  current_user.music_service.build_all_schools_and_users_xls(:filename => temp_filename)
  FileUtils.mv temp_filename, @filename
end
@spawn_id = spawner.handle

Upvotes: 0

raviolicode
raviolicode

Reputation: 2175

I think that the best approach is to use Resque to import and convert in a worker separately from the request.

Suppose you have a controller to add the Excel file, which I'm going to call Information model:

class InformationController < ApplicationController
  def create
    @information = Information.new(params[:information])
    if @information.save
      resque = Resque.enqueue(ImportDataJob, @information.id)
      redirect_to @information, :notice => "Successfully created information for further processing."
    else
      render :new
    end
  end
end

You'll need make a job, in this case ImportDataJob:

class ImportDataJob
  def self.perform(information_id)
    information = Information.find(information_id)
    # convert information.raw_csv or wherever attribute you saved the Excel or CSV into
    # and save it into the database where you need to
  end
end

You'll find a full tutorial in Resque RailsCast, where it shows how to add Resque into your existing Rails app.

Note: There's a conflict between README and the actual implementation for Resque. Apparently they want to change the way Resque is called (which is in the readme), but is not implemented yet. See this Issue in Github for more details.

Upvotes: 1

Related Questions