CODEWITHSUNDEEP
rubyherokumessage-queuesidekiq
purpletonic
purpletonic

Reputation: 1878

Shared message queues between apps with Heroku

I have two web applications, both to be hosted on Heroku. One is the front-end app, in the sense that all consumer facing actions will be through this app. New sign-ups, etc. The other is the back-end, this ones deals with all of the processing of new customers, their orders, and associated business-logic. I need these apps to talk to each-other and pass data between them. I'd rather not write an api on the BE app, as only the FE app would be consuming it, and this would mean I'd need to spend time writing security rules and restrictions to prevent others from accessing it.

My question is whether I can use a shared messaging queue between them instead, and whether this was a good idea, and something that's possible on Heroku? Does an add-on/service exist for this use-case? Most of the queuing add-ons listed have the workers and task creators in the same app.

I've been looking at Sidekiq as a possibility, as the BE app will need a queue for other tasks anyway, and this seems like a good candidate, but I'm not sure that even if I provide a shared Redis instance between the apps, Sidekiq has the capability to pass messages between them. Anyone know?

Upvotes: 2

Views: 959

Answers (3)

harlow
harlow

Reputation: 864

If each application has its own Redis connection you can push from one application to another Sidekiq environment by setting up another connection:

class RemoteWorker
  cattr_writer :client

  def self.push(worker_class_name, args, queue = 'default')
    client.push(
      'args' => [args],
      'class' => worker_class_name,
      'queue' => queue,
    )
  end

  def self.client
    @@client or raise 'Make sure to set client to Sidekiq::Client'
  end
end

Then configure it from config/initializers/remote_worker.rb

redis_config = YAML.load_file(Rails.root.join('config', 'redis.OTHER.yml'))[Rails.env]
redis = Redis.new(redis_config)
RemoteWorker.client = Sidekiq::Client.new(ConnectionPool.new{ redis })

Upvotes: 0

Henley Wing Chiu
Henley Wing Chiu

Reputation: 22515

Sidekiq works well in a distributed environment such as yours. Since all the information needed to process jobs resides in 1 single Redis server, you can have as many servers working on any type of job.

Upvotes: 1

wless1
wless1

Reputation: 3549

I've been having a lot of success using AMQP on Heroku. There's a heroku plugin available (CloudAMQP) and good ruby support for AMQP.

In terms of provisioning services like this on Heroku in general, I've been provisioning the plugin in the main Heroku application, and sharing the environment config with other applications.

Upvotes: 4

Related Questions