CODEWITHSUNDEEP
ruby-on-railsasynchronoussidekiq
ivan
ivan

Reputation: 6322

Sidekiq: perform_async and order-dependent operations

There's a controller action in my Rails app that contacts a user via text-message and email. For reasons I won't go into, the text-message needs to complete before the email can be sent successfully. I originally had something like this:

controller:

class MyController < ApplicationController
  def contact_user
    ContactUserWorker.perform_async(@user.id)
  end
end

workers:

class ContactUserWorker
  include Sidekiq::Worker

  def perform(user_id)
    SendUserTextWorker.perform_async(user_id)
    SendUserEmailWorker.perform_async(user_id)
  end
end

class SendUserTextWorker
  include Sidekiq::Worker

  def perform(user_id)
    user = User.find(user_id)
    user.send_text
  end
end

class SendUserEmailWorker
  include Sidekiq::Worker

  def perform(user_id)
    user = User.find(user_id)
    user.send_email
  end
end

This was unreliable; sometimes the email would fail, sometimes both would fail. I'm trying to determine whether perform_async was the cause of the problem. Was the async part allowing the email to fire off before the text had completed? I'm a little fuzzy on how exactly perform_async works, but that sounded like a reasonable guess.

At first, I refactored ContactUserWorker to:

class ContactUserWorker
  include Sidekiq::Worker

  def perform(user_id)
    user = User.find(user_id)
    User.send_text

    SendUserEmailWorker.perform_async(user_id)
  end
end

Eventually though, I just moved the call to send_text out of the workers altogether and into the controller:

class MyController < ApplicationController
  def contact_user
    @user.send_text
    SendUserEmailWorker.perform_async(@user.id)
  end
end

This is a simplified version of the real code, but that's the gist of it. It seems to be working fine now, though I still wonder whether the problem was Sidekiq-related or if something else was going on.

I'm curious whether my original structure would've worked if I'd used perform instead of perform_async for all the calls except the email call. Like this:

class MyController < ApplicationController
  def contact_user
    ContactUserWorker.perform(@user.id)
  end
end

class ContactUserWorker
  include Sidekiq::Worker

  def perform(user_id)
    SendUserTextWorker.perform(user_id)
    SendUserEmailWorker.perform_async(user_id)
  end
end

Upvotes: 2

Views: 10977

Answers (2)

dre-hh
dre-hh

Reputation: 8044

I'm curious whether my original structure would've worked if I'd used perform instead of perform_async for all the calls except the email call

It would have. But this is not what you actually intdending. What you really want is this:

class MyController < ApplicationController
  def contact_user
    ContactUserWorker.perform_async(@user.id)
  end
 end

class ContactUserWorker
  include Sidekiq::Worker
  attr_reader :user_id

 def perform(user_id)
  @user_id = user_id
  user.send_text
  user.send_email
 end

 def user
   @user ||= User.find user_id
 end

end

The problem was indeed the perform async part. It schedules both tasks to be executed in the background by a separate sidekiq daemon process. i guess your sidekiq is configured to execute the jobs concurrently. In the first version you've first scheduled the ContactUserWorker to perform it's job in a background outside of the current rails request. As this worker is startet later on, it kicks off two separate delayed workers in turn, which are then run in parallel and so there is no way to determine which of the both executes/finishes first.

I don't know what you mean exatly by sending text, but sending an email is an io blocking process and therefore it was a good idea to perform this in a background, because it would be blocking a complete rails process otherwise until the email is delivered (on a typical unicorn/passenger multi-process deployment). And as you actually want to execute both tasks sequentially and as an atomic operation, it's totally fine, performing them by a single sidekiq job/worker.

You also don't have to check if send_text succeeds. Sidekiq will retry the complete job if any part of it fails with an exception

Upvotes: 1

Sixty4Bit
Sixty4Bit

Reputation: 13402

If the email can only be sent after the text message has been sent, then send the email after successful completion of sending the text.

class ContactUserWorker
  include Sidekiq::Worker

  def perform(user_id)
    SendUserTextWorker.perform_async(user_id)
  end
end

class SendUserTextWorker
  include Sidekiq::Worker

  def perform(user_id)
    user = User.find(user_id)
    text_sent = user.send_text
    SendUserEmailWorker.perform_async(user_id) if text_sent
  end
end

class SendUserEmailWorker
  include Sidekiq::Worker

  def perform(user_id)
    user = User.find(user_id)
    user.send_email
  end
end

In user.send_text you need to handle the fact that neither the text or the email has been sent.

Upvotes: 2

Related Questions