CODEWITHSUNDEEP
rubyfastercsv
mkirk
mkirk

Reputation: 4040

How to remove repeated columns using ruby FasterCSV

I'm using Ruby 1.8 and FasterCSV.

The csv file I'm reading in has several repeated columns. 

| acct_id | amount | acct_num | color | acct_id | acct_type | acct_num |
|     345 |  12.34 |      123 |   red |     345 | 'savings' |      123 |
|     678 |  11.34 |      432 | green |     678 | 'savings' |      432 |

...etc

I'd like to condense it to:

| acct_id | amount | acct_num | color | acct_type |
|     345 |  12.34 |      123 |   red | 'savings' |
|     678 |  11.34 |      432 | green | 'savings' |

Is there a general purpose way to do this?

Currently my solution is something like:

headers = CSV.read_line(file)
headers = CSV.read_line # get rid of garbage line between headers and data
FasterCSV.filter(file, :headers => headers) do |row|
  row.delete(6) #delete second acct_num field
  row.delete(4) #delete second acct_id field

  # additional processing on the data
  row['color'] = color_to_number(row['color'])
  row['acct_type'] = acct_type_to_number(row['acct_type'])
end

Upvotes: 1

Views: 788

Answers (1)

steenslag
steenslag

Reputation: 80065

Assuming you want to get rid of the hardcoded deletions

  row.delete(6) #delete second acct_num field
  row.delete(4) #delete second acct_id field

Can be replaced by

row = row.to_hash

This will clobber duplicates. The rest of the posted code will keep working.

Upvotes: 1

Related Questions