CODEWITHSUNDEEP
ruby-on-railsrubyruby-on-rails-3testingrspec
user12882
user12882

Reputation: 4792

How can I speed up the creation of 5,000 records for my rspec tests?

I am using Ruby on Rails 3.2.2, FactoryGirl 3.1.0, FactoryGirlRails 3.1.0, Rspec 2.9.0 and RspecRails 2.9.0. In order to test my application I have to create a lot of records (about 5000) in the database, but that operation is very slow (it takes more than 10 minutes to create records). I proceed like this:

before(:each) do
  5000.times do
    FactoryGirl.create(:article,)
  end
end

How can I improve my spec code so to go faster?

Note: Maybe the slowness is given by (5) article callbacks that run before and after each article creation process, but I can skip those (since the only things I have to test are articles and not associated models) if those slow creation of records... is it possible to make that and is it the right way to proceed?

Upvotes: 9

Views: 3781

Answers (3)

Todd A. Jacobs
Todd A. Jacobs

Reputation: 84343

As others have mentioned, factories aren't supposed to be high-volume. Why would any reasonable unit test need 5,000 records?

That said, part of the reason for the slow-down is that #create saves each record to the database after it's built. You probably want to use FactoryGirl.build instead; see Using Factories.

Of course, you're going to be trading an I/O-bound process for a memory-bound process. If that's not what you really want, consider using #build to instantiate just enough records to perform each individual test, or using a fixture file optimized for bulk loads.

Upvotes: 0

Michael Durrant
Michael Durrant

Reputation: 96454

When you start processing large number of records like that, it's better to do it in sql directly to avoid the ActiveRecord overhead that you don't need.
I would look to write a script or create a stored procedure to do this dependig on your sql implementation. It'll quite likely finish in a few seconds.

Upvotes: 2

Ismael
Ismael

Reputation: 16720

I'm no expert in testing, but I think FactoryGirl isn't supposed to have this use. It should be to create just some records.

Think if you really need so many records for testing, if you do, do they have to be "reseted" before each test? (yes, you are creating them before each test, so you are creating 5000*number_of_tests records actually). Isn't a raw sql script a choice? So when your test db gets created all those records get created too.

Upvotes: 0

Related Questions