CODEWITHSUNDEEP
rubyconcurrencysucker-punch
sschmeck
sschmeck

Reputation: 7715

Wait for SuckerPunch::Job task completion

I try the Sucker Punch gem to process tasks in parallel. But I found no documentation how to wait for its termination. 

require 'sucker_punch'

class SuckerJob
  include SuckerPunch::Job
  workers 4

  def perform(event)
    sleep(rand(5))
    puts "[#{Thread.current.object_id}] End processing event   #{event}."
  end
end

10.times { |i| SuckerJob.perform_async(i) }
puts "Shutting down ..."
SuckerPunch::Queue.shutdown_all
puts "Shutdown finished, status: #{SuckerPunch::Queue.stats[SuckerJob.name]}"
# [Ugly] call internal method
SuckerPunch::Queue::QUEUES.fetch_or_store(SuckerJob.name).wait_for_termination(10)
puts "Wait finished, status: #{SuckerPunch::Queue.stats[SuckerJob.name]}"

It seems, that SuckerPunch::Queue.shutdown_all() returns before all tasks are completed.

Shutting down ...
[17487240] End processing event   1.
[17488760] End processing event   0.
[17487240] End processing event   4.
[17488760] End processing event   5.
[17486120] End processing event   2.
[17484940] End processing event   3.
[17487240] End processing event   6.
Shutdown finished, status: {"workers"=>{"total"=>3, "busy"=>3, "idle"=>0}, "jobs"=>{"processed"=>7, "failed"=>0, "enqueued"=>0}}
[17484940] End processing event   9.
[17488760] End processing event   7.
[17486120] End processing event   8.
Wait finished, status: {"workers"=>{"total"=>0, "busy"=>0, "idle"=>0}, "jobs"=>{"processed"=>10, "failed"=>0, "enqueued"=>0}}

How can I wait until all tasks are completed?

Upvotes: 3

Views: 953

Answers (2)

AlejandroVD
AlejandroVD

Reputation: 1822

I use this:

def wait_for_jobs(job_name:, count:, max_seconds: 100)
  Rails.logger.info "Waiting up to #{max_seconds} seconds for #{count} jobs to run"
  wait_time = 0
  while wait_time < max_seconds
    stats = SuckerPunch::Queue.stats[job_name]
    processed = stats['jobs']['processed']
    break unless processed < count

    sleep(1)
    wait_time += 1
  end
  raise StandardError, "Timeout while waiting for #{count} jobs of #{job_name} to have run!" unless wait_time < max_seconds

  Rails.logger.info "#{count} jobs took #{wait_time} seconds to run"
end

Upvotes: 0

Rajarshi Das
Rajarshi Das

Reputation: 12340

You can check status

or check stats you can do 

all_stats = SuckerPunch::Queue.stats

then 

stats = all_stats[SuckerJob.to_s]

then stats you get now you can see 

stats["jobs"]["processed"]

you can check like below 

  stats["jobs"]["processed"] > 0
  stats["jobs"]["failed"] == 0
   stats["jobs"]["enqueued"] == 0

Upvotes: 3

Related Questions