CODEWITHSUNDEEP
ruby-on-railscsverror-handling
JHFirestarter
JHFirestarter

Reputation: 83

Rescue CSV::MalformedCsvError: Illegal quoting in line n

Seems a common issue to have a buggy CSV file when attempting to parse to an array, AR model import, etc. I haven't found a working solution other than open in MS Excel and save as every day (not good enough!).

In a 60,000 row externally-provided, daily-updated csv file, there's an error: CSV::MalformedCSVError: Illegal quoting in line 95. (as an example). I'm happy to skip/forget the malformed row (i.e. it has only 1/60000th importance).

First attempt is to use CSV.foreach or similar, and simply begin rescue next end to skip the error. No dice. I wish this SO accepted answer was a wee bit more verbose: CSV.read Illegal quoting in line x (i.e. "just read the file yourself"--I think I tried that below).

And this SO Q&A (How can I further process the line of data that causes the Ruby FasterCSV library to throw a MalformedCSVError?) seems to have promise, but the accepted answer does not...quite...work in my seemingly similar case (modified for example clarity), executed via rake task:

file_path = "filename.csv"
my_array = []

File.open(file_path).each do |line| # `foreach` instead of `open..each` does the same
  begin     
    CSV.parse(line) do |row|
      my_array << row
    end
  rescue CSV::MalformedCSVError => er
    puts er.message
    counter += 1
    next
  end
  counter += 1
  puts "#{counter} read success"
end

Output =>

1 read success
2 read success
...
94 read success
Illegal quoting in line 1 # strange that it says `line 1` vs `95`, which may be the crux of what I do not understand here (e.g. some kind of look ahead error)
96 read success
...
60000 read success 
  # I checked using `line.inspect`, and the 60000th row is indeed being read/inspected
rake aborted!

CSV:MalformedCSVError: Illegal quoting in line 95

Upvotes: 5

Views: 4632

Answers (1)

Ravi Sankar Raju
Ravi Sankar Raju

Reputation: 2950

Your solution works. The expected result resides in the variable my_array.

Upvotes: 1

Related Questions