Rails 5 - Rake task to import data from CSV file

Question

I am trying to upload data I have kept in xls to my rails db. I have tried several different tutorials to get this working and outline the problems Im getting stuck with on each of them below. I don't mind which method I ultimately use, I'm just running out of ideas for how to try to solve this problem with any of these methods.

Importing CSV files by RichonRails

model:

class Randd::Field < ApplicationRecord

    require 'csv'


  def self.import(file)
    CSV.foreach(file.path, headers: true) do | row |
            randd_field_hash = row.to_hash
            randd_field = Randd::Field.where(id: randd_field_hash["id"])
            if randd_field.count == 1
                randd_field.first.update_attributes(randd_field_hash)
            else
                Randd::Fields.create!(randd_field_hash)
            # else
            # Randd::Field.create! row.to_hash
    end
  end
end


end

routes:

namespace :randd do
resources :fields do
      collection {post :import }#do
        # post :create
      end
end

Controller:

class Randd::FieldsController < ApplicationController

def index
    @randd_fields = Randd::Field.all
end

def new
    @field = Randd::Field.new
end

def create
    # @field = Randd::Field.new(randd_field_params)
    Randd::Field.create(params[:randd_field][:file])
    redirect_to action: "index", notice: "Data imported"
end

def show
    redirect_to action: "index"
end

def import
    Randd::Field.import(params[:file])
    # Randd::Field.create(params[:randd_field]['file'])
    redirect_to action: "index", notice: "Data imported"
end

private
 def randd_field_params
  params.fetch(:randd_field, {}).permit(:title, :anz_reference)
 end

end

When I save this and try it, I get an error that says:

undefined method `path' for nil:NilClass

The error message points at the import method in the model

GoRails Tutorial:

I tried to follow the GoRails tutorial for importing data from a csv file.

In my rails app, I have saved the file as for_codes.csv.

In my lib/tasks folder, I have made a file called: import.rake

In my input.rake file, I have:

require 'csv'
namespace :import do

  desc "import research fields from csv"
  task rannd_fields: :environment do
    filename = File.join Rails.root, 'for_codes.csv'
    counter = 0

    CSV.foreach(filename, headers: true) do | row |
      p row ["anz_reference"]
      for_code = Randd::Field.create(anz_reference: row["anz_reference"], title: title)
      counter += 1 if randd_field.persisted?
    end
    puts "Imported #{counter} field codes"
  end
end

When I then try this in the console, with:

bundle exec rake import:randd_fields

I get an error that says:

NameError: undefined local variable or method `randd_fields' for main:Object

The model I'm trying to make records on is called:

class Randd::Field < ApplicationRecord

The table in the database is called:

randd_fields

Can anyone see what I need to do to rake this task?

BACKTRACE

bundle exec rake import:rannd_fields
rake aborted!
ArgumentError: invalid byte sequence in UTF-8
/Users/live/lib/tasks/import.rake:9:in `block (2 levels) in <top (required)>'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'
/Users/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
/Users/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => import:rannd_fields
(See full trace by running task with --trace)
MacBook-Pro:live em$ bundle exec rake import:rannd_fields --trace
** Invoke import:rannd_fields (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute import:rannd_fields
rake aborted!
ArgumentError: invalid byte sequence in UTF-8
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:2016:in `=~'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:2016:in `init_separators'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:1538:in `initialize'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:1273:in `new'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:1273:in `open'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/csv.rb:1130:in `foreach'
/Users/live/lib/tasks/import.rake:9:in `block (2 levels) in <top (required)>'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:248:in `block in execute'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:243:in `each'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:243:in `execute'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:187:in `block in invoke_with_call_chain'
/Users/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:180:in `invoke_with_call_chain'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/task.rb:173:in `invoke'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:152:in `invoke_task'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:108:in `block (2 levels) in top_level'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:108:in `each'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:108:in `block in top_level'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:117:in `run_with_threads'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:102:in `top_level'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:80:in `block in run'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:178:in `standard_exception_handling'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/lib/rake/application.rb:77:in `run'
/Users/.rvm/gems/ruby-2.3.1@global/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'
/Users/.rvm/gems/ruby-2.3.1@global/bin/rake:23:in `load'
/Users/.rvm/gems/ruby-2.3.1@global/bin/rake:23:in `<main>'
/Users/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
/Users/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => import:rannd_fields

NEXT ATTEMPT

I next tried making a brand new file in my rails app. I typed these three lines in:

anz_reference | title 010104p | Combinatorics and Discrete Mathematics (excl. Physical Combinatorics) 010107p | Mathematical Logic, Set Theory, Lattices and Universal Algebra

The first row are the headers. They match attributes on the Randd::Field model.

The next two rows are the content Im trying to import. The cell values are separated by the | (as shown in the GoRails video).

When I try this, I get the same error as I posted above:

$ bundle exec rake import:randd_fields --trace ** Invoke import:randd_fields (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute import:randd_fields nil rake aborted! NameError: undefined local variable or method `title' for main:Object

Can anyone see what I'm doing wrong?

I have also tried this tutorial How to import CSV files in Rails by Matt Morgante and tried to use the rails cast and the Roo gem to the same end. Neither of these options work - the Rails Cast is old, so probably out of date, and the comments in the post to the other tutorial show many others have not been able to get that approach to work - maybe things in Rails have changed since it was proposed.

It was suggested on the GoRails forum that the problem might be attributable to the CSV file is not encoded to the UTF-8 specifications. It was suggested that this might be overcome by transferring the data to a sheet in google drive and then downloading this as a csv for use with rails. I tried this.

I get the same error as I got when I ran the rake task with a file created in xls.

ZHONG'S SUGGESTION

Taking Zhong's suggestion, I tried changing the import.rake task to:

require 'csv'
namespace :import do

  desc "import research fields from csv"
  task randd_fields: :environment do
    filename = File.join Rails.root, 'for_codes.csv'
    counter = 0

    CSV.foreach(filename, headers: true) do | row |
      p row ["anz_reference"]
      # for_code = Randd::Field.create(anz_reference: row["anz_reference"], title: title)
      for_code = Randd::Field.create(anz_reference: row["anz_reference"], title: row['title'])
      counter += 1 if for_code.persisted?
    end
    puts "Imported #{counter} field codes"
  end
end

When I then try to

bundle exec rake import:randd_fields

I get the same error:

bundle exec rake import:randd_fields
NameError: undefined local variable or method `randd_fields' for main:Object
    from (irb):6

Answer 1

There might be two problems here.

1. Encoding in your csv file. ArgumentError: invalid byte sequence in UTF-8

2. Undefined local variable. NameError: undefined local variable or method 'randd_fields' for main:Object

   I guess you are trying to count created/imported records:

   for_code = Randd::Field.create(anz_reference: row["anz_reference"], title: title)
   counter += 1 if randd_field.persisted?
   

   the record you created is for_code, however you are checking against randd_field. This should fix it

   counter +=1 if for_code.presisted?
   

Updated:

$ bundle exec rake import:randd_fields --trace ** Invoke import:randd_fields (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute import:randd_fields nil rake aborted! NameError: undefined local variable or method `title' for main:Object

this is because title variable is not defined. I guess you want to use row[] here.

for_code = Randd::Field.create(anz_reference: row["anz_reference"], title: row['title'])

Updated 2:

You have a typo in your rake task name

Updated 3:

I think you are calling bundle exec rake import:randd_fields inside rails console. Run it directly in terminal should fix it.