CODEWITHSUNDEEP
ruby-on-railsminitestruby-on-rails-6
gbarillot
gbarillot

Reputation: 329

Websockets flaky tests after upgrading to Rails 6

While upgrading a middle size app from Rails 5.1 to 6.0.2.2 (+ Ruby 2.6.1 -> 2.6.3) I start to get flaky tests in all kind of tests. 90% coverage, 500 hundreds tests, and between 0 to 8 tests failings, totally random. After a long bug hunt, I noticed that I got everything working with 100% confidence if I skip all Websocket related tests.

This is typically what I have to test and how I'm doing it (Minitest/Spec syntaxe):

class Api::PlayersControllerTest < ActionController::TestCase
  before do
    @user = users(:admin)
    sign_in @user
  end

  it "broadcasts stop to user's player" do
    put :update, format: :json, params: {id: @user.id}

    assert_broadcast_on("PlayersChannel_#{@user.id}", action: "stop")
  end
end

Notice that it's not an "integration test" because we're using a raw API call. What I have to check is: if some request is coming to some controller, ActionCable is broadcasting a Websocket message. That's why I have to use Devise sign_in helper in a controller Test.

I have 23 tests like this one, and they all used to pass without any problem using Rails 5.1. Started one by one using a focus, they also all pass 100% both in Rails 5 or 6. The problem is when executing the whole test suite, I start to get flakiness in all sections (Unit/Models tests included), mostly related to dataset consistency. Actually, it looks like fixtures are not (or poorly) reloaded.

Any ideas? Is it something wrong with what I'm doing, or do you think it's a Rails 6 issue?

Upvotes: 2

Views: 165

Answers (1)

gbarillot
gbarillot

Reputation: 329

Ok, solved by adding Database Cleaner Gem. Pretty curious that it works using the same ":transaction" strategy that's in use with the Rails barebone fixture management... but anyway, it works!

Upvotes: 2

Related Questions