CODEWITHSUNDEEP
ruby-on-railsrubymultithreadingactiverecord
Wes
Wes

Reputation: 764

Using ruby-thread, sidekiq, and saving with ActiveRecord using Thread.pool

I'm attempting to save something to my database via ActiveRecord using ruby-thread's Thread.pool in a Sidekiq job. The odd thing is that the code works fine when executing it manually in my Rails console, but it hangs, without any error, when it is run in Sidekiq.

Here is an example of what I am doing:

require 'thread/pool'

@array_job = ["first", "second", "third", "forth"]

def do_this(something)
  ...
  #do stuff
  ...
  @result = {key1: value1, key2: value2, key3: value3}
end

def save_result
  save_this = ResultTable.new
  save_this.column1 = @result[key1]
  save_this.column2 = @result[key2]
  save_this.column3 = @result[key3]
  save_this.save
end

pool = Thread.pool(8)

@array_job.each_with_index do |something|
  pool.process do
    do_this(something)
    save_result
  end
end

pool.shutdown

The idea is that I want to execute everything in @array_job concurrently, and once the execution finishes, I want to save the result to the database table. I don't want to save the results to an array and later call on that array in order to save them to a table - I want the table save to happen immediately after do_this(something) has executed for each job.

When I run this in Sidekiq, the Sidekiq job will hang indefinitely on save_this = ResultTable.new and never recovers until I kill the Sidekiq instance itself. As stated before, if I run this manually in the rails console, it works fine. It also works if I remove pool.process do and run each job one at a time (but that of course is not something I want to do - I need the jobs to run concurrently).

Does anyone know why the Sidekiq job would hang when it gets to saving to the ResultTable using thread/pool? I'm using the latest version of Sidekiq, Ruby 4.2.0, and Rails 5.0.1.

Upvotes: 2

Views: 555

Answers (1)

Wes
Wes

Reputation: 764

Figured out that this was caused by a deadlock in the dev environment, which explains why it worked in the rails console but not within the dev environment of my rails app. Appears to be specific to Rails 5. It's explained here.

Upvotes: 1

Related Questions