Capistrano, Passenger deployment failing

Question

There is this app which has been upgraded from Rails 3 to 4. (In addition to upgraded from Capistrano 2 to 3). This is an app that was built, has worked fine for a couple years and now we are adding some features and updating things - but deployment is failing.

I've tried reinstalling gems, passenger apache module, various capistrano deployment configuration options (with sudo, without sudo, setting specific gem paths with default_env ) and working through various Capistrano upgrade documentation, guides, blog posts and other SO posts...

Deployment errors

When attempting to push the changes to the staging server the deployment keeps failing.

Here's the base error when running cap staging deploy:

/dependency.rb:315:in `to_specs': Could not find 'passenger' (>= 0) among 14 total gem(s) (Gem::LoadError)

There must be some simple conflict / confusion with paths / sudo / rvmsudo and has taken more time than I'd like to admit debugging...

Initially I tried doing everything without sudo but wasn't getting success. According to the rvm documentation (https://rvm.io/integration/sudo) I tried editng the sudoers file.

So following rvm.io documentation I comment out Default secure_path=... and add Default env_keep and the following error occurs:

Exception while executing as user@example.org: sudo exit status: 127
sudo stdout: /usr/bin/env: ruby: No such file or directory

gem env

Output of $ gem env on the staging server:

    $ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 2.4.6
  - RUBY VERSION: 2.0.0 (2015-02-25 patchlevel 643) [x86_64-linux]
  - INSTALLATION DIRECTORY: /home/user/.rvm/gems/ruby-2.0.0-p643
  - RUBY EXECUTABLE: /home/user/.rvm/rubies/ruby-2.0.0-p643/bin/ruby
  - EXECUTABLE DIRECTORY: /home/user/.rvm/gems/ruby-2.0.0-p643/bin
  - SPEC CACHE DIRECTORY: /home/user/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /home/user/.rvm/rubies/ruby-2.0.0-p643/etc
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-linux
  - GEM PATHS:
     - /home/user/.rvm/gems/ruby-2.0.0-p643
     - /home/user/.rvm/gems/ruby-2.0.0-p643@global
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /home/user/.rvm/gems/ruby-2.0.0-p643/bin
     - /home/user/.rvm/gems/ruby-2.0.0-p643@global/bin
     - /home/user/.rvm/rubies/ruby-2.0.0-p643/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin
     - /usr/games
     - /usr/local/games
     - /home/user/.rvm/bin
     - /home/user/.rvm/bin

config/deploy/staging.rb

# server-based syntax
# ======================
# Defines a single server with a list of roles and multiple properties.
# You can define all roles on a single server, or split them:

# server 'example.com', user: 'deploy', roles: %w{app db web}, my_property: :my_value
# server 'example.com', user: 'deploy', roles: %w{app web}, other_property: :other_value
# server 'db.example.com', user: 'deploy', roles: %w{db}

set :stage, :staging

server "example.org", user: "user", roles: %w{web, app, db}


# role-based syntax
# ==================

# Defines a role with one or multiple servers. The primary server in each
# group is considered to be the first unless any  hosts have the primary
# property set. Specify the username and a domain or IP for the server.
# Don't use `:all`, it's a meta role.

role :app, %w{example.org}
role :web, %w{example.org}
role :db,  %w{example.org}
# role :app, %w{deploy@example.com}, my_property: :my_value
# role :web, %w{user1@primary.com user2@additional.com}, other_property: :other_value
# role :db,  %w{deploy@example.com}



# Configuration
# =============
# You can set any configuration variable like in config/deploy.rb
# These variables are then only loaded and set in this stage.
# For available Capistrano configuration variables see the documentation page.
# http://capistranorb.com/documentation/getting-started/configuration/
# Feel free to add new variables to customise your setup.



# Custom SSH Options
# ==================
# You may pass any option but keep in mind that net/ssh understands a
# limited set of options, consult the Net::SSH documentation.
# http://net-ssh.github.io/net-ssh/classes/Net/SSH.html#method-c-start
#
# Global options
# --------------
#  set :ssh_options, {
#    keys: %w(/home/rlisowski/.ssh/id_rsa),
#    forward_agent: false,
#    auth_methods: %w(password)
#  }
#
# The server-based syntax can be used to override options:
# ------------------------------------
# server 'example.com',
#   user: 'user_name',
#   roles: %w{web app},
#   ssh_options: {
#     user: 'user_name', # overrides user setting above
#     keys: %w(/home/user_name/.ssh/id_rsa),
#     forward_agent: false,
#     auth_methods: %w(publickey password)
#     # password: 'please use keys'
#   }


set :rails_env, "staging"

set :application, "res.example.org"
set :repo_url,  "user@example.org:/home/user/webdocs/appname.git"
# set :repo_url,  "ssh://user@example.org:3699/home/user/webdocs/appname.git"


set :ssh_options, {
  port: 3699,
  verbose: :debug 
}

# set :rvm_map_bins, %w{gem rake ruby bundle rvmsudo}
set :rvm_map_bins, fetch(:rvm_map_bins, []).push('rvmsudo')
# set :rbenv_ruby, "2.0.0" 
set :port, 3699
set :scm, :git
set :pty, true
# set :use_sudo, true
# Or: `accurev`, `bzr`, `cvs`, `darcs`, `git`, `mercurial`, `perforce`, `subversion` or `none`
# set :user, 'user'
set :use_sudo, false #true
set :deploy_to, "/var/www/res"
set :branch, "staging"
set :deploy_via, :remote_cache

set :passenger_rvm_ruby_version, "2.0.0"

set :default_env, { 
      rvm_bin_path: '~/.rvm/bin',
      path: "/home/user/.rvm/gems/ruby-2.0.0-p643/bin:/home/user/.rvm/rubies/ruby-2.0.0-p643/bin/ruby:/home/user/.rvm/gems/ruby-2.0.0-p643/bin:$PATH" 
    }

set :passenger_restart_with_sudo, true
set :log_level, :debug
# set :passenger_restart_with_sudo, true
set :passenger_environment_variables, { :path => '/home/user/.rvm/gems/ruby-2.0.0-p643/bin:/home/user/.rvm/rubies/ruby-2.0.0-p643/bin/ruby:/home/user/.rvm/gems/ruby-2.0.0-p643/bin:$PATH' }



=begin 
set :default_environment, {
  'PATH' => 'home/user/.rvm/gems/ruby-1.9.3-p385/bin:/home/user/.rvm/gems/ruby-1.9.3-p385@global/bin:/home/user/.rvm/rubies/ruby-1.9.3-p385/bin:/home/user/.rvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/user/.rvm/bin:$PATH'
  #'PATH' => '/home/user/.rvm/gems/default/bin:/home/user/.rvm/gems/default@global/bin:/home/user/.rvm/rubies/default/bin:/home/user/.rvm/bin:$PATH'  
  #'PATH' => '/home/user/.rvm/gems/ruby-1.9.3-p385/bin:/home/user/.rvm/gems/ruby-1.9.3-p385@global/bin:/home/user/.rvm/rubies/ruby-1.9.3-p385/bin:/home/user/.rvm/bin:$PATH'  
}
=end

#default_environment["GEM_PATH"] ="/home/user/.rvm/gems/ruby-1.9.2-p320"
# /home/bb/.rvm/gems/ruby-1.9.2-p318/bin:/home/bb/.rvm/gems/ruby-1.9.2-p318@global/bin:/home/bb/.rvm/rubies/ruby-1.9.2-p318/bin:/home/bb/.rvm/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/bb/.cabal/bin:/home/bb/.xmonad/bin 
#default_environment["PATH"] = "/home/bb/.rvm/gems/ruby-1.9.2-p318/bin:/home/bb/.rvm/gems/ruby-1.9.2-p318@global/bin:/home/bb/.rvm/rubies/ruby-1.9.2-p318/bin:/home/bb/.rvm/bin:$PATH"


# role :web, "example.org"                          # Your HTTP server, Apache/etc
#role :app, "example.org"                          # This may be the same as your `Web` server
#role :db,  "example.org", :primary => true # This is where Rails migrations will run
#role :db,  "example.org"

# if you're still using the script/reaper helper you will need
# these http://github.com/rails/irs_process_scripts

#after "deploy", "deploy:bundle_gems"
#after "deploy:bundle_gems", "deploy:restart"

# If you are using Passenger mod_rails uncomment this:
namespace :deploy do

  desc "Install gems"
    task :bundle_gems do
    on roles(:app), in: :sequence, wait: 5 do
       run "cd #{deploy_to}/current && bundle install"
     end
   end

  desc "Setup sym links for uploads..."
  task :make_links do
    on roles(:app), in: :sequence, wait: 5 do
      run "ln -s #{shared_path}/uploads/ #{deploy_to}/current/public/uploads"
    end
  end

  desc "Precompile assets...."
  task :precompile_assets do
    on roles(:app), in: :sequence, wait: 5 do
      run "cd #{deploy_to}/current && bundle exec rake --trace assets:precompile RAILS_ENV=staging"
    end
  end

  # task :start do ; end
  # task :stop do ; end
=begin  
  desc "Restart App"
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      execute :touch, release_path.join('tmp/restart.txt')
      # run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
    end
  end
=end
end

after "deploy", "deploy:make_links"

after "deploy", "deploy:bundle_gems"

# after "deploy", "deploy:precompile_assets"

Capfile

# Load DSL and set up stages
require 'capistrano/setup'

# Include default deployment tasks
require 'capistrano/deploy'

    # Include tasks from other gems included in your Gemfile
    #
    # For documentation on these, see for example:
    #
    #   https://github.com/capistrano/rvm
    #   https://github.com/capistrano/rbenv
    #   https://github.com/capistrano/chruby
    #   https://github.com/capistrano/bundler
    #   https://github.com/capistrano/rails
    #   https://github.com/capistrano/passenger
    #
    require 'capistrano/rvm'
    # require 'capistrano/rbenv'
    # require 'capistrano/chruby'
    require 'capistrano/bundler'
    require 'capistrano/rails/assets'
    require 'capistrano/rails/migrations'
    require 'capistrano/passenger'

    # Load custom tasks from `lib/capistrano/tasks` if you have any defined
    Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }

deploy.rb

# config valid only for current version of Capistrano
lock '3.4.0'

set :application, 'appname'
set :repo_url, 'user@example.org:/home/user/webdocs/appname.git'

# Default branch is :master
# ask :branch, `git rev-parse --abbrev-ref HEAD`.chomp

# Default deploy_to directory is /var/www/my_app_name
# set :deploy_to, '/var/www/my_app_name'

# Default value for :scm is :git
set :scm, :git

# Default value for :format is :pretty
# set :format, :pretty

# Default value for :log_level is :debug
set :log_level, :debug

set :stages, %w(staging production)
set :default_stage, "staging"
# Default value for :pty is false
# set :pty, true

# Default value for :linked_files is []
# set :linked_files, fetch(:linked_files, []).push('config/database.yml', 'config/secrets.yml')

# Default value for linked_dirs is []
# set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'vendor/bundle', 'public/system')

# Default value for default_env is {}
# set :default_env, { path: "/opt/ruby/bin:$PATH" }

# Default value for keep_releases is 5
# set :keep_releases, 5

namespace :deploy do

  after :restart, :clear_cache do
    on roles(:web), in: :groups, limit: 3, wait: 10 do
      # Here we can do anything such as:
      # within release_path do
      #   execute :rake, 'cache:clear'
      # end
    end
  end

end

Any suggestions / direction appreciated!

Answer 1

After more testing and further reviewing the Passenger & Capistrano documentation (https://www.phusionpassenger.com/documentation/Users%20guide%20Apache.html & http://capistranorb.com/documentation/getting-started/authentication-and-authorisation/ ) there were a few points of confusion resolved.

The gem installation of passenger was uninstalled. Then following the Passenger documentation, and starting over from scratch, Passenger was installed via the Ubuntu repositories / apt-get.

In addition, passwordless sudo access needed to be setup for the deploy user - and update the Capistrano deployment file accordingly.

Specifically making sure set :passenger_restart_with_sudo, true was set

So the Could not find 'passenger' (>= 0) error is now gone.

Capistrano deployment is getting further. Now it's just hanging up on my rake tasks which is a different issue to work out.