Sidekiq async tasks are done synchronously

Question

I have two Sidekiq workers:

# app/workers/client_worker.rb
class ClientWorker
  include Sidekiq::Worker

  def perform(*args)
    puts "client started\n"
  end
end

and

# app/workers/server_worker.rb
class ServerWorker
  include Sidekiq::Worker

  def perform(*args)
    i = 0
    loop do
      begin
        i = i + 1
        puts "server waiting"
        sleep(2)
        if i > 5
          break
        end
      rescue => e
      end
    end
  end
end

I have two Rails apps, same rails version, same ruby version, same gems versions.

rvm use ruby-2.4.2@rails5.2

On a fresh Rails App When I start server worker, then immediatly after, client worker, I get client message as expected amongst server messages:

2018-06-21T22:44:19.855Z 32364 TID-gn863tph0 ServerWorker JID-7cdd1fd3d6fe63c91d99f3b0 INFO: start
server waiting
server waiting
server waiting
2018-06-21T22:44:24.356Z 32364 TID-gn865q1zk ClientWorker JID-7dc887d6824d0b789afec69d INFO: start
client started
2018-06-21T22:44:24.357Z 32364 TID-gn865q1zk ClientWorker JID-7dc887d6824d0b789afec69d INFO: done: 0.001 sec
server waiting
server waiting
server waiting
2018-06-21T22:44:31.858Z 32364 TID-gn863tph0 ServerWorker JID-7cdd1fd3d6fe63c91d99f3b0 INFO: done: 12.003 sec

On an already developed Rails app

When I repeat this process, tasks are running one after the other:

2018-06-21T22:41:34.250Z 32247 TID-grdaosqf3 ServerWorker JID-5a2d9917e6575d6858644474 INFO: start
server waiting
server waiting
server waiting
2018-06-21T22:41:38.402Z 32247 TID-grdamch6b ClientWorker JID-5da83582991bf284f6357046 INFO: start
server waiting
server waiting
server waiting
client started
2018-06-21T22:41:46.259Z 32247 TID-grdaosqf3 ServerWorker JID-5a2d9917e6575d6858644474 INFO: done: 12.007 sec
2018-06-21T22:41:46.259Z 32247 TID-grdamch6b ClientWorker JID-5da83582991bf284f6357046 INFO: done: 7.858 sec

What I've tried

• I have removed all installed gems in the old app, so that the old Rails app Gemfile ressembles the new one. I have removed all custom libraries, all jobs, all custom configurations

• I have compared Sidekiq.options on both apps, they are the same:

  {:queues=>[], :labels=>[], :concurrency=>25, :require=>".", :environment=>nil, :timeout=>8, :poll_interval_average=>nil, :average_scheduled_poll_interval=>5, :error_handlers=>[], :death_handlers=>[], :lifecycle_events=>{:startup=>[], :quiet=>[], :shutdown=>[], :heartbeat=>[]}, :dead_max_jobs=>10000, :dead_timeout_in_seconds=>15552000, :reloader=>#}

Now I'm running out of ideas! Can anyone suggest me a method to find out what's wrong?

EDIT Okey, after a long hesitation I decided to do the same change I made in new app in old app, namely: in new app I remarked that workers are not included automatically, so I added the following initializer:

# config/initializers/01_extensions.rb
Dir["#{Rails.root}/app/workers/*.rb"].each { |file| require file }

In old app, workers were included automatically (I really don't know which is the default behaviour!), so this initializer was not necessary.

BUT NOW when I decided to add the same initializer to old app, tasks are running async as expected.

Now I have two questions:

• do we need to include sidekiq workers explicitly? (not mentionned in doc)
• what doest the explicit including of workers change in the sidekiq behaviour?

Answer 1

In development, Rails 5 effectively limits you to one job at a time, even with multiple threads. This global "executor" lock allows for the development reloading that Rails is famous for.

In production, jobs will be executed concurrently as you'd expect.