CODEWITHSUNDEEP
ruby-on-railsrubypriority-queuesidekiq
user984621
user984621

Reputation: 48443

Sidekiq - how to execute the job immediately (+ does make sense to use a queue in this case)?

I have a task that I need to generate immediately after the request is created and get it done ASAP.

So for this purpose, I have created a /config/sidekiq.yml file where I defined this:

---
:queues:
  - default
  - [critical, 10]

And for the respective worker, I set this:

class GeneratePDFWorker
  include Sidekiq::Worker
  sidekiq_options queue: 'critical', retry: false

  def perform(order_id)
    ...

Then, when I call this worker:

GeneratePDFWorker.perform_async(@order.id)

So I am testing this. But - I found this post, where is said that if I want to execute the tasks immediately, I should call:

GeneratePDFWorker.new.perform(@order.id)

So my question is - should I use the combination of a (critical) queue + the new (GeneratePDFWorker.new.perform) method? Does it make sense?

Also, how can I verify that the tasks is execute as critical?

Thank you

Upvotes: 2

Views: 6281

Answers (3)

Anthony E
Anthony E

Reputation: 11235

As Walking Wiki mentioned, GeneratePDFWorker.new.perform(@order.id) will call the worker synchronously. So if you did this from a controller action, the request would block until the perform method completed.

I think your approach of using priority queues for critical tasks with Sidekiq is the way to go. As long as you have enough Sidekiq workers, and your queue isn't backlogged, the task should run almost immediately so the benefit of running your worker in-process is pretty much nil. So I'd say yes, it does make sense to queue in this case.

Also, you're probably aware of this, but sidekiq has a great monitoring UI: https://github.com/mperham/sidekiq/wiki/Monitoring. This should should make it easy to get reliable, detailed metrics on the performance of your workers.

Upvotes: 1

Shiva
Shiva

Reputation: 12514

should I use the combination of a (critical) queue?

Me:

Yes you can use critical queue if you feel so. A queue with a weight of 2 will be checked twice as often as a queue with a weight of 1.

Tips:

  • Keep the number of queues fewer as possible. Sidekiq is not designed to handler tremendous number of queues.
  • Also keep weights as simple as possible. If you want queues always processed in a specific order, just declare them in order without weights.

the new (GeneratePDFWorker.new.perform) method?

Me: No, using sidekiq in the same thread asynchronously is bad in the first place. This will hamper your application's performance as your application-server will be busy for longer. This will be very expensive for you. Then what will be the point of using sidekiq?

Upvotes: 0

Walking Wiki
Walking Wiki

Reputation: 699

So my question is - should I use the combination of a (critical) queue + the new (GeneratePDFWorker.new.perform) method? Does it make sense?

Using GeneratePDFWorker.new.perform will run the code right there and then, like normal, inline code (in a blocking manner, not async). You can't define a queue, because it's not being queued.

Upvotes: 2

Related Questions