CODEWITHSUNDEEP
ruby-on-rails-3sidekiq
Ovesh
Ovesh

Reputation: 5379

Is it possible to run sidekiq on a separate host from the rails host?

I'd like to setup sidekiq so that it runs on a separate host from my rails app, to avoid weighing down the web server when running memory-intensive background tasks.

What are the configuration details I need to look at?

Are there known best practices or configurations for this?

EDIT:

To clarify, what I mean to ask is, how do I configure rails so that sidekiq API calls (MyWorker.perform_async) would run if the sidekiq process isn't running locally?

Upvotes: 18

Views: 6559

Answers (1)

bioneuralnet
bioneuralnet

Reputation: 5301

Edit to clarify

Before everything else, you should understand that when you run perform_async, it simply sticks the Jobs into a Redis queue. Sidekiq watches this queue and runs jobs when they come in. So as long as your app and your Sidekiq are looking at the same Redis server (and same database, of course), things will work.

Original answer

First, as you can probably guess, it will need a duplicate checkout of your code so that it can run. No big deal.

Secondly, it will need access to your database and redis servers on the other box. That means making sure those ports are open on the other server. That can get tricky, b/c you ideally don't want those exposed to the open Internet. Usually, for a multi-box setup, you'll have only one box exposed to the open Internet. It communicates with the rest of your boxes over private IPs:

Public Web server

Runs Apache/Nginx and maybe your app servers.

Private App Server(s) (optional)

Runs your app servers, if they aren't running on the Public server. Connects to your Database and Redis server.

Private Sidekiq Server(s) (optional)

Runs Sidekiq. Connects to your Database and Redis server.

Private Database/Redis Server

Runs database and Redis. Of course they can split out to different servers if required.

Upvotes: 25

Related Questions