CODEWITHSUNDEEP
ruby-on-railsrubycsv
Henley Wing Chiu
Henley Wing Chiu

Reputation: 22515

How to convert array of ActiveRecord models to CSV?

I got an array of ActiveRecord models that I wish to convert to a CSV. I tried researching gems like FasterCSV, but they just seem to work with strings and arrays, not ActiveRecord models.

In short, I want to convert:

user1 = User.first
user2 = User.last
a = [user1, user2]

TO:

   id,username,bio,email
    1,user1,user 1 bio,user1 email
    1,user2,user 2 bio,user2 email

Is there an easy Rails way to do this?

Upvotes: 60

Views: 35022

Answers (10)

Tilo
Tilo

Reputation: 33732

If you use the smarter_csv gem, it automatically tracks all attributes, e.g. when they are set only sparsely:

items = [
  {data: {a: 1, b: 2, c: 3}},
  {data: {b: 4, c: 5, d: 6}},
  {data: {c: 7, d: 8, e: 9}},
  {data: {x: 0, y: 0, z: 0}},
].map{|x| OpenStruct.new(x)}

SmarterCSV.generate(file) do |writer|
  writer << items.map(&:data)
end

generates:

 cat /tmp/out.csv
a,b,c,d,e,x,y,z
1,2,3
,4,5,6
,,7,8,9
,,,,,0,0,0

This example is showing how to do this for JSONB columns, or general array of hashes, where not all keys may be known in advance.

Upvotes: 0

ccorey
ccorey

Reputation: 3

I made a method in the ApplicationRecord that allows you to make a csv sting from any Model, it also only uses the headers from the select.

class ApplicationRecord < ActiveRecord::Base
  class << self
    def to_csv_string
      CSV.generate do |csv|
        csv << self.first.as_json.keys
        self.find_each do |selfie|
          csv << selfie.attributes.values
        end
      end
    end
  end
end
# User.select(:id, :name, :email).to_csv_string
# => "id,fname,lname\n1,Chet,Corey\n"

Hope this is useful

Upvotes: 0

tomb
tomb

Reputation: 1436

I had this same problem and combined a couple of these answers so I could call to_csv on a model or relation, then input a file name and create a csv file.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  def self.to_csv
    require 'csv'

    p "What is the name of your file? (don't forget .csv at the end)"

    file_name = gets.chomp

    CSV.open("#{file_name}", "wb") do |csv|
      csv << column_names
      all.find_each do |model|
        csv << model.attributes.values_at(*column_names)
      end
    end
  end
end

Now from console you can call .to_csv on any model or any db query or activerecord relation.

Upvotes: 1

rudolph9
rudolph9

Reputation: 8119

The following will write the attributes of all users to a file:

CSV.open("path/to/file.csv", "wb") do |csv|
  csv << User.attribute_names
  User.find_each do |user|
    csv << user.attributes.values
  end
end

Similarly you could create a CSV string:

csv_string = CSV.generate do |csv|
  csv << User.attribute_names
  User.find_each do |user|
    csv << user.attributes.values
  end
end

Upvotes: 123

theterminalguy
theterminalguy

Reputation: 1941

This might be off the original question but solve the problem. If you plan to make all or some of your Active Record models be able to convert to csv, you can use ActiveRecord concern. An example is shown below

module Csvable
  extend ActiveSupport::Concern 

  class_methods do
    def to_csv(*attributes)
      CSV.generate(headers: true) do |csv| 
        csv << attributes 

        all.each do |record| 
          csv << attributes.map { |attr| record.send(attr) }
        end 
      end
    end
  end
end

The attribute provided will be used as the header for the CSV and it is expected that this attribute corresponds to methods name in the included class. Then you can include it in any ActiveRecord class of your choice, in this case, the User class 

class User 
  include Csvable 

end

Usage

User.where(id: [1, 2, 4]).to_csv(:id, :name, :age)

Note: This only works for ActiveRecord relation and not for arrays

Upvotes: 5

Yuki Inoue
Yuki Inoue

Reputation: 3777

Yet another similar answer, but here's what I usually do.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  def self.to_csv
    CSV.generate do |csv|
      csv << column_names
      all.find_each do |model|
        csv << model.attributes.values_at(*column_names)
      end
    end
  end
end

Instead of hacking existing module, I'd usually put this code in the ApplicationRecord class, the base class of all the models (usually).

If any further elaboration is needed, I'd add a named parameter to the to_csv method, and handle those features as much as possible in this class.

This way, the to_csv method will be available to both Model and its Relation. E.g.

User.where(role: :customer).to_csv
# => gets the csv string of user whose role is :customer

Upvotes: 2

Adobe
Adobe

Reputation: 13477

One can also utilize the sql engine for this. E.g. for sqlite3:

cat << EOF > lib/tasks/export-submissions.sql
.mode      csv
.separator ',' "\n"
.header    on


.once "submissions.csv"

select
  *
from submissions
;
EOF

sqlite3 -init lib/tasks/export-submissions.sql db/development.sqlite3 .exit

If you are on CentOS 7 -- it ships with sqlite released in 2013. That version did not know separator and once yet. So you might need to download the latest binary from the web-site: https://sqlite.org/download.html install it locally, and use the full path to the local installation:

~/.local/bin/sqlite3 -init lib/tasks/export-submissions.sql db/development.sqlite3 .exit

Upvotes: 0

Mario Olivio Flores
Mario Olivio Flores

Reputation: 2795

If you need something quick and dirty, not so much for production as just grabbing some data for a non-technical user, you could paste this in console:

require 'csv'
class ActiveRecord::Relation
  def to_csv
    ::CSV.generate do |csv|
      csv << self.model.attribute_names
      self.each do |record|
        csv << record.attributes.values
      end
    end
  end
end

Then do: User.select(:id,:name).all.to_csv

If you were going to production, I'd probably turn this into a decorator around ActiveRecord::Relation and more precisely ensuring that the order of your fields/attributes.

Upvotes: 4

Steven Barrag&#225;n
Steven Barrag&#225;n

Reputation: 1244

with julia_builder you can configure a csv export pretty easily.

class UserCsv < Julia::Builder
  # specify column's header and value
  column 'Birthday', :dob
  # header equals 'Birthday' and the value will be on `user.dbo`

  # when header and value are the same, no need to duplicate it.
  column :name
  # header equals 'name', value will be `user.name`

  # when you need to do some extra work on the value you can pass a proc.
  column 'Full name', -> { "#{ name.capitalize } #{ last_name.capitalize }" }

  # or you can pass a block
  column 'Type' do |user|
    user.class.name
  end
end

and then

users = User.all
UserCsv.build(users)

Upvotes: 2

Brian
Brian

Reputation: 31282

@rudolph9's answer is really awesome. I just want to leave a note for people who need to do this task periodically: making it as a rake task would be a good idea!

lib/tasks/users_to_csv.rake

# usage:
# rake csv:users:all => export all users to ./user.csv
# rake csv:users:range start=1757 offset=1957 => export users whose id are between 1757 and 1957
# rake csv:users:last number=3   => export last 3 users
require 'csv' # according to your settings, you may or may not need this line

namespace :csv do
  namespace :users do
    desc "export all users to a csv file"
    task :all => :environment do
      export_to_csv User.all
    end

    desc "export users whose id are within a range to a csv file"
    task :range => :environment do |task, args|
      export_to_csv User.where("id >= ? and id < ?", ENV['start'], ENV['offset'])
    end

    desc "export last #number users to a csv file"
    task :last => :environment do |task, arg|
      export_to_csv User.last(ENV['number'].to_i)
    end

    def export_to_csv(users)
      CSV.open("./user.csv", "wb") do |csv|
        csv << User.attribute_names
        users.each do |user|
          csv << user.attributes.values
        end
      end
    end
  end
end

Upvotes: 12

Related Questions