CODEWITHSUNDEEP
ruby-on-railsrubyresqueresque-scheduler
Krish
Krish

Reputation: 477

How can I start recurring background jobs when a user visits a web page?

I am working on creating a Rails web application with background workers for performing some of the tasks in the background on a set interval. I am using Resque with Redis for queuing the background jobs, and using Resque-scheduler to run it on a set interval for ex., every 30 seconds or so.

The background job needs to be enqueued only when a user visits a particular page, and it should run on a schedule until the user moves away from that page. Basically, I would like to set the schedule dynamically during runtime. My application is deployed in Cloud, and the main Rails app and the background workers run as a separate processes communicating through Redis. How can I set the schedule dynamically, and from where?

resque_schedule.yml

do_my_job:
  every: 1m
  class: BackgroundJob
  description: Runs the perform method in MyJob
  queue: backgroundq

inside the controller

  def index
    Resque.enqueue(BackgroundJob)
  end

background_job.rb

class BackgroundJob
  @queue = :backgroundq

  @job_helper = BackgroundHelper.new

  def self.perform
    @job_helper.get_job_data
  end

end

Upvotes: 0

Views: 486

Answers (1)

Kristján
Kristján

Reputation: 18803

Resque-scheduler can do what you're describing using dynamic schedules. Every time a user visits your page, you create a schedule with Resque.set_schedule that runs every 30 seconds on that user. When the user leaves, you abort with Resque.remove_schedule. I can imagine a lot of ways the user might leave the page without you noticing (worst case, what if their computer loses power?), so I'd be worried about the schedules being left around. I'm also not sure how happy resque-scheduler remains as you add more and more schedules, so you might run into trouble with lots of users.

You could minimize the number of schedules by having one job that runs every 30 seconds over every user you currently need running, like:

class BackgroundJob
  def self.perform
    User.active.each { ... }
  end
end

If you do something like set User#last_seen_at when the user visits your page and have User.active load everyone seen in the past 10 minutes, then you'll be running every 30 seconds on all your active users, and there's no chance of a user's jobs lasting well after they leave, because it times out in 10 minutes. However, if you have more users than you can do work for in 30 seconds, the scheduled job won't be finished before its next copy starts up, and there will be trouble. You might be able to get around that by having the top-level job enqueue for each user a secondary job that actually does the work. Then as long as you can do that in 30 seconds and have enough resque workers for your job volume, things should be fine.

A third way to approach it is to have the user's job enqueue another copy of itself, like so:

class BackgroundJob
  def self.perform(user_id)
    do_work(user_id)
    Resque.enqueue_in(30.seconds, BackgroundJob, user_id)
  end
end

You enqueue the first job when the user visits your page, and then it keeps itself going. This way is nice because every user's job run independently, and you don't need either a separate schedule for each or a top-level schedule managing everything. You'll still need a way to stop the jobs running once the user is gone, perhaps with a timeout or TTL counter.

Finally, I'd question whether Resque is the right tool for your task. If you want something happening every 30 seconds as long as a user is on a page, presumably you want the user seeing it happen, so you should consider implementing it in the browser. For example, you could set up a 30 second JavaScript interval to hit an API endpoint that does some work and returns the value. This avoids any need for Resque at all, and will automatically stop when the user navigates away, even if they lose power and your code doesn't get a chance to clean up.

Upvotes: 1

Related Questions