Running rails specs using rake command => 0 fails, while running them with rspec command => 1 warning 5 fails

Question

When I run the specs of my Rails project, I usually use rake, which outputs the following:

josh@macbuech:~/Documents/Work/MuheimWebdesign/base/src (master *)$ rake
/Users/josh/.rvm/rubies/ruby-2.1.0/bin/ruby -I/Users/josh/.rvm/gems/ruby-2.1.0@base/gems/rspec-core-3.1.7/lib:/Users/josh/.rvm/gems/ruby-2.1.0@base/gems/rspec-support-3.1.2/lib /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/rspec-core-3.1.7/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb  --color --format Fuubar
Run options: include {:focus=>true}                                             

All examples were filtered out; ignoring {:focus=>true}
 121/121 |======================= 100 ========================>| Time: 00:00:07 

Pending:
  File upload uses an uploaded file (from the temporary cache) after a re-display and then successful submit of the form
    # Seems to be an issue of Carrierwave, see https://github.com/carrierwaveuploader/carrierwave/issues/1414
    # ./spec/features/file_upload_spec.rb:41

Finished in 7.05 seconds (files took 5.82 seconds to load)
121 examples, 0 failures, 1 pending

Top 3 slowest examples (1.76 seconds, 25.0% of total time):
  RegistrationsController POST 'create' valid input creates a guest user and converts it to a registered one
    0.76552 seconds ./spec/controllers/registrations_controller_spec.rb:9
  Signing out signs the user out
    0.53429 seconds ./spec/features/user/sessions/destroy_spec.rb:9
  File upload uses an uploaded file (from the temporary cache) after a re-display and then successful submit of the form
    0.4628 seconds ./spec/features/file_upload_spec.rb:41

Top 3 slowest example groups:
  Signing out
    0.53429 seconds average (0.53429 seconds / 1 example) ./spec/features/user/sessions/destroy_spec.rb:3
  RegistrationsController
    0.41286 seconds average (0.82572 seconds / 2 examples) ./spec/controllers/registrations_controller_spec.rb:3
  File upload
    0.35467 seconds average (2.13 seconds / 6 examples) ./spec/features/file_upload_spec.rb:3

Randomized with seed 56456

0 failures.

But when I run it with the rspec command, I get this:

josh@macbuech:~/Documents/Work/MuheimWebdesign/base/src (master *)$ rspec
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
......................*.............................................../Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activerecord-4.2.0/lib/active_record/associations.rb:1693: warning: already initialized constant Role::HABTM_Users
/Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activerecord-4.2.0/lib/active_record/associations.rb:1693: warning: previous definition of HABTM_Users was here
/Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activerecord-4.2.0/lib/active_record/associations.rb:1693: warning: already initialized constant User::HABTM_Roles
/Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activerecord-4.2.0/lib/active_record/associations.rb:1693: warning: previous definition of HABTM_Roles was here
......FFFF.............F...........................

Pending:
  File upload uses an uploaded file (from the temporary cache) after a re-display and then successful submit of the form
    # Seems to be an issue of Carrierwave, see https://github.com/carrierwaveuploader/carrierwave/issues/1414
    # ./spec/features/file_upload_spec.rb:41

Failures:

  1) User creating a user validates uniqueness of name
     Failure/Error: expect(@user).to have(1).error_on(:name)
       expected 1 error on :name, got 2
     # ./spec/models/user_spec.rb:90:in `block (3 levels) in <top (required)>'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `block in load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

  2) User creating a user validates presence of email
     Failure/Error: expect(@user).to have(1).error_on(:email)
       expected 1 error on :email, got 2
     # ./spec/models/user_spec.rb:95:in `block (3 levels) in <top (required)>'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `block in load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

  3) User creating a user validates presence of password
     Failure/Error: expect(@user).to have(1).error_on(:password)
       expected 1 error on :password, got 2
     # ./spec/models/user_spec.rb:100:in `block (3 levels) in <top (required)>'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `block in load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

  4) User creating a user validates presence of name
     Failure/Error: expect(@user).to have(1).error_on(:name)
       expected 1 error on :name, got 2
     # ./spec/models/user_spec.rb:83:in `block (3 levels) in <top (required)>'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `block in load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

  5) User creating a guest validates presence of name
     Failure/Error: expect(@guest).to have(1).error_on(:name)
       expected 1 error on :name, got 2
     # ./spec/models/user_spec.rb:43:in `block (3 levels) in <top (required)>'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `block in load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:268:in `load'
     # /Users/josh/.rvm/gems/ruby-2.1.0@base/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

Finished in 7.34 seconds (files took 0.47745 seconds to load)
121 examples, 5 failures, 1 pending

Failed examples:

rspec ./spec/models/user_spec.rb:86 # User creating a user validates uniqueness of name
rspec ./spec/models/user_spec.rb:93 # User creating a user validates presence of email
rspec ./spec/models/user_spec.rb:98 # User creating a user validates presence of password
rspec ./spec/models/user_spec.rb:81 # User creating a user validates presence of name
rspec ./spec/models/user_spec.rb:40 # User creating a guest validates presence of name

Top 3 slowest examples (2.18 seconds, 29.7% of total time):
  Signing up signs up a new user
    1.07 seconds ./spec/features/user/registration/new_spec.rb:4
  I18n offers contents in english
    0.66078 seconds ./spec/features/i18n_spec.rb:10
  File upload displays a preview of an uploaded file (from the temporary cache) after a re-display of the form
    0.44416 seconds ./spec/features/file_upload_spec.rb:27

Top 3 slowest example groups:
  Signing up
    1.07 seconds average (1.07 seconds / 1 example) ./spec/features/user/registration/new_spec.rb:3
  File upload
    0.34402 seconds average (2.06 seconds / 6 examples) ./spec/features/file_upload_spec.rb:3
  I18n
    0.25478 seconds average (0.76435 seconds / 3 examples) ./spec/features/i18n_spec.rb:3

Randomized with seed 49596

1 warning and 5 failures, and the failures seem have to do with the warning (at least this is a possibility). It looks like some association is loaded twice, which results in the errors in the specs.

How can I debug this situation? Because there seem to be so many possibilities, it's hard to post more specific code snippets here. But here's the link to my GitHub repo, maybe someone can reproduce it?

https://github.com/jmuheim/base

Answer 1

I debug the app and found out:

1) Your specs depend on order. For example

bin/rspec spec --seed 49796 # gives 5 failures
bin/rspec spec --seed 1     # gives 0 failure

2) Also I place caller to User and I can see that model User is loaded twice (you should stop spring to see the first calling). Analyzing caller stack (for seed 49796) I noticed that second one is different and has strange lines

...
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:84:in `block in project_models'", 
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:83:in `each'", 
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:83:in `project_models'", 
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:61:in `block (3 levels) in <top (required)>'", 
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:60:in `each'", 
"/home/vaska/projects/base/spec/i18n_keys_spec.rb:60:in `block (2 levels) in <top (required)>'",
...

This line loads models second time. When I comment lines 86-88 in i18n_keys_spec all three cases passes successfully (I mean bin/rake, bin/rspec spec 1, bin/rspec spec --seed 49796).

I am not sure that removing these lines are solution because I suspect that spring caches the app and that is why it works now. Maybe you can check if class defined before require model file (for example spring itself checks it). From other hand i18n_keys_spec loads rails_helper which should load all app including models.

Hope it helps you