CODEWITHSUNDEEP
ruby-on-railsrubymultithreading
Yu Yan
Yu Yan

Reputation: 11

Rails hang in Thread

I am trying to make a single-threaded part multi-threaded. The initial single thread code works fine, but after I put it into a thread, the server hangs without any log information in the console.

Initial version:

data_array.each do |data|
  service = SomeService.new(data)
  data['answers'] = service.answer_data
end

Multi-threaded version:

threads = []
threads << Thread.new {
  data_array.each do |data|
    service = SomeService.new(data)
    data['answers'] = service.answer_data
  end
}
threads.each(&:join)

I tried binding.pry and get some more information:

  1. I have a Class called AggregationReport < Report
  2. When SomeService tried to run AggregationReport.new, it's stuck there forever.

Please help me understand what is happening here, any thoughts/help would be much appreciated!

Ruby version: Ruby27x Rails: 5.2

Upvotes: 1

Views: 1002

Answers (1)

melcher
melcher

Reputation: 1601

I'm making some assumptions about why you're threading this, but I'd expect typical threading code to follow the following basic structure:

threads = data_array.map do |data|
  Thread.new { SomeService.new(data).answer_data }
end

answers = threads.map(&:value)

For each piece of work you need done:

  1. Create a thread that does the work and stores the result in a threadsafe way
  2. Join each thread to ensure execution (this is handled by #value)
  3. Collect the results

If your service needs a database connection (e.g. postgres, redis, etc) you'll need to manage this as well. For activerecord add ActiveRecord::Base.with_connection{ } inside the thread to allocate a db connection. There are similar approaches for Redis/etc. Then ensure you have sufficient connections for the # of threads you want to create (look at the pool: setting in config/database.yml).

Thread.new { ActiveRecord::Base.with_connection{ SomeService.new(data).answer_data } }

For this and other reasons, it isn't always a great idea to spawn N threads, so instead of this approach you may consider using a thread pool of X and ensure you have a db connection pool of at least X+1 connections. All of this can be complex. There are great gems that sit on top of ruby threading and make it easier to do this exact kind of thing. I'd strongly recommend checking out concurrent-ruby (bundled with Rails).

Upvotes: 3

Related Questions