Why does Capistrano deployment fail at assets:precompile without error?

Question

I've tried everything I could think of, but I can't get deployment to work.

The app needs to deploy to a VPS hosted by Alwaysdata and running ruby 2.6.2, using Capistrano for deployment. It's a Rails 6.0.2.2 application, using webpack for JS and sprockets for legacy scripts, all images and CSS.

# On local machine (MacOS)

$ bundle exec cap production deploy
00:00 git:wrapper
00:01 git:check
00:03 deploy:check:directories
00:03 deploy:check:linked_dirs
00:03 deploy:check:make_linked_dirs
00:05 git:clone
00:06 git:update
00:08 git:create_release
00:10 deploy:set_current_revision
00:10 deploy:symlink:linked_files
00:12 deploy:symlink:linked_dirs
00:18 deploy:config:bundler
00:20 bundler:install
00:20 deploy:assets:precompile
#<Thread:0x00007fb35ba959f0@/Users/Goulven/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/sshkit-1.21.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
/Users/Goulven/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/sshkit-1.21.0/lib/sshkit/command.rb:97:in `exit_status=': rake exit status: 1 (SSHKit::Command::Failed)
rake stdout: Nothing written
rake stderr: Nothing written

  INFO [3def24f1] Running bundle exec rake assets:precompile as vtcontrol@ssh-vtcontrol.alwaysdata.net

 DEBUG [3def24f1] Command: cd /home/www/app/releases/20200409174918 && ( export NODE_ENVIRONMENT="production" RAILS_ENV="production" RAILS_GROUPS="" ; bundle exec rake assets:precompile )

Running the last command locally or on the server does not produce any errors, only warnings:

# On production server:

$ cd /home/www/app/releases/20200409174918 && ( export NODE_ENVIRONMENT="production" RAILS_ENV="production" RAILS_GROUPS="" ; bundle exec rake assets:precompile )
yarn install v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.2.12: The platform "linux" is incompatible with this module.
info "fsevents@1.2.12" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > webpack-dev-server@3.10.3" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "webpack-dev-server > webpack-dev-middleware@3.7.2" has unmet peer dependency "webpack@^4.0.0".
success Already up-to-date.
Done in 0.62s.

Hypothesis I've thought of:

1. Running out of RAM. Happens mainly with Digital Ocean, provider excludes that possibility, and anyway it would not compile when sshing to the server.
2. An issue with node vs nodejs. This used to nag me but it no longer happens, at first because I used sprockets only and added the mini_racer gem, which packages node for ExecJS, then because I included Bootstrap using webpacker instead of Sprockets, thus removing the dependency on autoprefixer-rails which errors out when an old version of nodejs is present alongside node.
3. Capistrano incorrectly interpreting yarn/webpacker warnings for failures. I extracted Bootstrap into its own app/javascripts/bootstrap.js because webpacker complained that the generated JS was too large, so this can be ruled out. Yarn complains about unmet peer dependencies but these are dev dependencies, I don't think it should matter. This might be the problem, but how can I test that?
4. Anything else I should try? Could using webpack and sprockets cause conflicts in production for instance? It works fine in development, and the generated assets should not overwrite one another.

---

Here are the relevant portions of my Gemfile:

# Gemfile

# Use SCSS for stylesheets
gem 'sass-rails', '>= 6'
# Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker
gem 'webpacker', '~> 4.0'

group :production do
  # Used to load env vars from an .env file during deploy
  gem 'dotenv-rails'
end

group :development do
  gem 'capistrano', '~> 3.13.0', require: false
  gem 'capistrano-rails', '~> 1.4', require: false
  gem 'capistrano-bundler', '~> 1.6', require: false
  gem 'capistrano-rails-console', require: false
end

And finally the relevant portion of my Capfile and deploy.rb:

# Capfile

require 'capistrano/bundler'
require 'capistrano/rails'
require 'capistrano/rails/assets'
require 'capistrano/rails/migrations'
require 'capistrano/rails/console'
require 'dotenv'
Dotenv.load

# config/deploy.rb

# Setup bundler
# Unset capistrano/bundler default flags because Bundler has deprecated them
set :bundle_flags,      '--quiet'
set :bundle_path,       nil
set :bundle_without,    nil
before 'bundler:install', 'deploy:config:bundler'

# Remove gems no longer used to reduce disk space used
# This command requires loading capistrano/bundler in Capfile
after 'deploy:published', 'bundler:clean'

# Skip migration if files in db/migrate were not modified
# This command requires loading capistrano/rails in Capfile
set :conditionally_migrate, true

# Rails app server manages the database
set :migration_role, :app

# Defaults to nil (no asset cleanup is performed)
# If you use Rails 4+ and you'd like to clean up old assets after each deploy,
# set this to the number of versions to keep
set :keep_assets, 2

Thanks for your help!

Answer 1

Thanks for pointing me in the right direction. I had the same error and it turned out to be a memory issue. For anyone with the same issues, check if you ran out of memory with free -m during compilation. If you ran out you'll see something like -bash: fork: Cannot allocate memory.

I'm using Digitalocean and my solution was to add swap space: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04

Answer 2

Okay, after days of trying everything I could think of I tried something I'd already tested but with the correct syntax this time, I guess, because it works now.

Here is what I needed to add to Capistrano's config file to ensure Yarn was available when deploying:

# In deploy.rb
# Add or adjust default_env to append .npm-packages to $PATH:
set :default_env, {
   PATH: '$HOME/.npm-packages/bin/:$PATH',
   NODE_ENVIRONMENT: 'production'
}

Explanation: VPS users are allowed to install binaries like Yarn "globally" (using npm install --global yarn). Under-the-hood the binary is installed in $HOME/.npm-packages/bin, and this folder is added to the $PATH for interactive/login shell sessions. Since Capistrano does its utmost to NOT pick up this, we have to force-feed the updated $PATH in deploy.rb.