CODEWITHSUNDEEP
ruby-on-railsiterator
Chad Johnson
Chad Johnson

Reputation: 21895

Data iterators in Rails?

When I do

users = User.find(:all)

in Rails, from my understanding, it pulls in all users from my database.

This seems like it could easily eat your server's memory if you have thousands of user records. Is this true? If it is true, is there a way to iterate over data in a table without killing your server?

Upvotes: 1

Views: 169

Answers (3)

Nikita Barsukov
Nikita Barsukov

Reputation: 2984

Indeed, your method can easily consume a lot of server resources.

As an alternative to finding objects in batches, you can use will_paginate helper.

Upvotes: 1

bioneuralnet
bioneuralnet

Reputation: 5301

Yes, it's definitely true. There may be some "object pagination" plugins for that purpose, but I have never looked into it.

One thing I do know is that ORMs like ActiveRecord tend to be very wasteful by default. Let's say you only need your user's ids and usernames, but there are actually 12 columns in the users table. By default ActiveRecord grabs all 12 columns, putting unnecessary load on both your database and your server's memory. To eliminate that waste in Rails 3:

users = User.select('id, username').all

I think Rails 2 would look something like

users = User.find(:all, :select => 'id, username')

Maybe not exactly what you were looking for, but that's cut out quite a few bottlenecks of mine.

Upvotes: 2

tommasop
tommasop

Reputation: 18765

Since rails 2.3 you can use find_each and find_in_batches:

User.find_each { |user| user.some_method }

will load a default of 1000 users. You also have the :batch_size option to change the default size.

Find in batches is similar but it provides the array to the block instead of the single object:

 User.find_in_batches do |users|
  users.each { |user| user.some_method }
end

Upvotes: 5

Related Questions