CODEWITHSUNDEEP
ruby-on-railsperformancetimerspec
stk1234
stk1234

Reputation: 1116

Accurately Timing RSpec Specs

I have 2 tests. One with a database call and one that uses a double. I'd like to understand the performance difference between them. I realize this is likely to be pretty small, but I'm just trying to get a better sense of the latency aspect of the trade off when opting to use a stub or an actual call to the db.

I originally thought this would be as simple as running something like 

time rspec spec_with_db_spec.rb

and 

time rspec spec_with_double_spec.rb

independently and maybe a bunch of times over and over in random order so I could take the averages, however, I'm running into some issues.

  1. It seems like time might not be the best way, as it could happen to coincide with a particularly expensive process occurring on my computer at the same time. I suppose running it over and over could help with that, but I'm not sure.
  2. I realized I could maybe use RSpec's time output, but based on (albeit old) posts, it seems like that may not be reliable.

Does anyone have ideas for this? It doesn't need to be perfect, but just something I could run and see a difference and how much that difference approximately is (order of magnitude is fine).

Upvotes: 1

Views: 799

Answers (2)

Schwern
Schwern

Reputation: 164679

Using time on an rspec test will not produce accurate results. The time will include all the other things a test has to do: loading Ruby, loading gems, and the rspec framework. This can swamp the time the code you're trying to benchmark takes.

You'll have only run the test once, it must be run many times to get an accurate result. And by running the different tests in different processes at different times might mean they're subject to different circumstances on a machine. For example, what if one runs just as your anti-virus starts running?

Instead, use the built in Benchmark module. These will only count the time taken by the benchmarked code.

Upvotes: 2

max
max

Reputation: 101811

You can use Rubys built in Benchmark tool:

require 'rails_helper'
require 'benchmark' # from the stdlib

RSpec.describe 'Benchmark' do
  let(:create_bm) do
    Benchmark.bm { FactoryBot.create_list(:user, 100) } 
  end

  let(:build_bm) do
    Benchmark.bm { FactoryBot.build_list(:user, 100) } 
  end

  it 'can give you the total time' do
    pending "Creating the list took #{create_bm.total} seconds"
  end

  specify 'you do not need two separate spec files' do
    pending "Building the list took #{build_bm.total} seconds"
  end

  specify 'stubbing is faster' do
    expect(build_bm.total).to be < create_bm.total
  end
end

Upvotes: 3

Related Questions