CODEWITHSUNDEEP
ruby-on-railssidekiq
dandlezzz
dandlezzz

Reputation: 613

Sidekiq ActiveJob track spawned jobs in

A lot of time I will have background worker code that looks like this.

class MyJob < ActiveJob::Base

 def perform
   User.find_in_batches.each do |batch|
     MyOtherJobToProcessBatchOfUsers.perform_later(batch)
   end
 end
end

Basically, one jobs that spawns many child jobs. Is there an easy way to know when all of the jobs are finished? I know I could increment a counter in the parent job and have each child job decrement that counter. But as anyone who has tried to do that will tell you, you can run into some complications.

Any help would be appreciated.

Upvotes: 0

Views: 421

Answers (1)

Mike Perham
Mike Perham

Reputation: 22208

This is what Sidekiq Pro's Batch feature does. You are right: it's incredibly difficult to build correctly. I don't know of any open source library that implements it.

https://github.com/mperham/sidekiq/wiki/Batches http://sidekiq.org/pro/

Upvotes: 2

Related Questions