CODEWITHSUNDEEP
ruby-on-rails-3.2queuesidekiq
Jignesh Gohel
Jignesh Gohel

Reputation: 6552

Sidekiq worker using a a "custom" queue name gets assigned to "default" queue

Rails 3.2.9
Ruby 1.9.3dev (2011-09-23 revision 33323) [i686-linux]
sidekiq  2.10.1

If I define a worker like below with a custom queue name

  class BulkEmailWorker
    include Sidekiq::Worker
    # https://github.com/mperham/sidekiq/wiki/Advanced-Options
    sidekiq_options(queue: :bulk_mails, backtrace: true)

    # https://github.com/mperham/sidekiq/wiki/Best-Practices
    # Section: Make your jobs small and simple
    def perform
      # https://github.com/mperham/sidekiq/wiki/Delayed-Extensions
      scheduled_mails = MailTemplate.with_schedule
      scheduled_mails.each do |mail_template|
        BulkMailer.delay.general_mail(mail_template.id)
      end
    end
  end

and I invoke the worker like this:

  BulkEmailWorker.perform_async

it doesn't work (in other words doesn't send email).

Inspecting the data in redis I found following:

  redis 127.0.0.1:6379> keys *
  1) "myapp:stat:processed:2013-04-24"
  2) "myapp:queue:default"
  3) "myapp:stat:processed"
  4) "myapp:queues"

  redis 127.0.0.1:6379> lrange myapp:queue:default -100 100

   1) "{\"retry\":true,\"queue\":\"default\",\"timeout\":30,\"class\":\"Sidekiq::Extensions::DelayedMailer\",\"args\":[\"---\\n- !ruby/class 'BulkMailer'\\n- :general_mail\\n- - 1\\n\"],\"jid\":\"e46693944febf7ae26ec67a0\"}"

redis 127.0.0.1:6379>

As can be seen above the worker is assigned to queue "default" and I guess due to this the worker is unable to process.

I started sidekiq using following command for above scenario:

  bundle exec sidekiq -e dev_mysql -C config/sidekiq.yml -q bulk_mails

However when I remove from my worker following

    sidekiq_options(queue: :bulk_mails, backtrace: true)

and start sidekiq using following command:

  bundle exec sidekiq -e dev_mysql -C config/sidekiq.yml

it works (in other words it sends email successfully).

/config/sidekiq.yml

# https://github.com/mperham/sidekiq/wiki/Logging
# http://stackoverflow.com/questions/15260634/sidekiq-configuration-for-multiple-environments
---
  :verbose: true
  :pidfile: ./tmp/pids/sidekiq.pid
  :logfile: ./log/sidekiq.log
  :concurrency:  25

I have just started using Sidekiq and am novice to it.So please bear with me if I there is some concept which I misunderstood or unaware about.

I need to get rid of the mentioned problem so that my worker gets assigned to the desired queue and also processed successfully.

Thanks, Jignesh

Upvotes: 4

Views: 11037

Answers (2)

Romain Tribes
Romain Tribes

Reputation: 1818

In addition, note that you can push a job on the queue you want by using Sidekiq::Client.push instead of perform_async:

Sidekiq::Client.push({
  'class' => BulkEmailWorker,
  'queue' => 'bulk_mails',
  'args'  => [ 1, 2 ]
})

Note that the keys (class, queue, args) must be strings, not symbols, or you will get the ArgumentError: Message must include a class and set of arguments.

This is especially useful when you need to push a job on a queue with a variable name (bulk_mails.1, bulk_mails.2, etc.), so you can easily control concurrency by allowing only one job at a time on some queues, thanks to sidekiq-limit_fetch.

Upvotes: 19

zwippie
zwippie

Reputation: 15515

You can add the queue names and weights in the sidekiq.yml config file:

:queues:
- [bulk_mails, 7]
- [default, 5]

Start sidekiq with the -C config/sidekiq.yml option. Now you can refer to the queue name in your worker:

sidekiq_options :queue => :bulk_mails

Upvotes: 8

Related Questions