CODEWITHSUNDEEP
ruby-on-railsrspec
Joshua Muheim
Joshua Muheim

Reputation: 13195

When running all specs, some specs fail (sometimes), but when running only specific file, they pass (always)

When running $ rspec on my Rails 4.1 application, some of the specs in user_spec.rb fail:

104 examples, 5 failures, 2 pending

And all 5 errors are because there are two errors on a certain field instead of only one:

1) User creating a guest validates presence of name
   Failure/Error: expect(@guest).to have(1).error_on(:name)
     expected 1 error on :name, got 2
   # ./spec/models/user_spec.rb:43:in `block (3 levels) in <top (required)>'
   # -e:1:in `<main>'

The errors are always double: @messages={:name=>["can't be blank", "can't be blank"]}>, but should be like this: @messages={:name=>["can't be blank"]}>.

Strangely, this doesn't happen, when running only the specified file ($ rspec ./spec/models/user_spec.rb).

22 examples, 0 failures

I tried $ git bisect, but couldn't locate the problem.

So if anybody could please try to checkout https://github.com/jmuheim/communes/commit/0dee0d4983809bc19b6cb97ff9bab071fc866b02 and run the specs ($ rspec and $ rspec ./spec/models/user_spec.rb) and tell me whether the double errors are occurring, and maybe try to help me debugging the problem, I would be really thankful.

I suspect Spring application pre-loader could be part of the problem, because sometimes, after Spring is explicitly stopped ($ spring stop), $ rspec passes, and sometimes it doesn't. Can't locate a pattern, looks very random to me:

$ spring stop
Spring stopped.

$ rspec
Finished in 3.66 seconds
104 examples, 0 failures, 2 pending

$ spring stop
Spring stopped.

$ rspec
104 examples, 5 failures, 2 pending

I can't really reproduce this pattern. Very strange and annoying/frustrating. Any help is highly appreciated.

Upvotes: 4

Views: 2056

Answers (2)

Jeremiah
Jeremiah

Reputation: 199

This was the first answer I came across googling a similar problem so I wanted to add the fix I found that worked best to save others some searching.

rspec 3.3+ has a --bisect command that will determine the smallest subset of tests that will fail.

run it with a seed that is known to fail and let it churn out a small set of specs that you need to look at.

rspec --seed 1245 --bisect

https://relishapp.com/rspec/rspec-core/docs/command-line/bisect

Upvotes: 3

Frederick Cheung
Frederick Cheung

Reputation: 84114

What you have is a spec order dependency problem: your user specs fail if some of the other specs have run before. When running only the user specs, then they are ok. You've got rspec set to run specs in a random order (which is designed to expose this sort of problems) which is why the problem comes and goes.

It sounds like the problem is that the User class is being loaded a second time, which is causing the validations to be added a second time. When the specs fails rspec will output a line telling you what the seed for the ordering was: if you run the specs with the seed option rspec shows you then the order will remain constant, enabling you to home in on the problem.

One approach might be to put a breakpoint (or just output caller) right at the top of user.rb - it should then become evident if user.rb is being loaded more than once and what is causing it.

Upvotes: 1

Related Questions