MacOS Passenger Ruby PG Gem Segmentation Fault

Question

For some reason a segmentation fault keeps my site from even loading the first page. I've reinstalled rubies, updated gems, reinstalled postgresql and nginx, checked over my nginx.conf file, and perused logs. Currently I only have one Ruby 2.3.1 installed via RVM, and IRB works like a charm. Passenger says preloader found an error.

From the looks of it, it's using Ruby 2.0.0p648 interpreter but 2.3.1 is the only version installed. Can I change that somehow? I've set rvm to use and --default to 2.3.1, and that's what it shows, but it keeps coming back to 2.0.0 somehow. It seems RVM's settings won't update. I'm stumped.

    An error occurred while starting up the preloader. It exited before signalling successful startup back to Phusion Passenger. Please read this article for more information about this problem.
    Raw process output:

    /Users/rich/.rvm/gems/ruby-2.3.1/gems/pg-0.19.0/lib/pg_ext.bundle: [BUG] Segmentation fault
    ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]

    -- Crash Report log information --------------------------------------------
         See Crash Report log file under the one of following:
             * ~/Library/Logs/CrashReporter
             * /Library/Logs/CrashReporter
             * ~/Library/Logs/DiagnosticReports
             * /Library/Logs/DiagnosticReports
         the more detail of.

    -- Control frame information -----------------------------------------------
    c:0023 p:-17535534978622 s:0081 e:000080 TOP    [FINISH]
    c:0022 p:---- s:0079 e:000078 CFUNC  :require
    c:0021 p:0009 s:0075 e:000074 TOP    /Users/rich/.rvm/gems/ruby-2.3.1/gems/pg-0.19.0/lib/pg.rb:4 [FINISH]
    c:0020 p:---- s:0071 e:000070 CFUNC  :require
    c:0019 p:0039 s:0067 e:000066 BLOCK  /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:91 [FINISH]
    c:0018 p:---- s:0063 e:000062 CFUNC  :each
    c:0017 p:0055 s:0060 e:000059 BLOCK  /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:86 [FINISH]
    c:0016 p:---- s:0054 e:000053 CFUNC  :each
    c:0015 p:0032 s:0051 e:000050 METHOD /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:75
    c:0014 p:0013 s:0047 e:000046 METHOD /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler.rb:106
    c:0013 p:0027 s:0043 e:000042 TOP    /Library/WebServer/Documents/HQ/app.rb:7 [FINISH]
    c:0012 p:---- s:0041 e:000040 CFUNC  :require
    c:0011 p:0020 s:0037 e:000036 BLOCK  config.ru:5 [FINISH]
    c:0010 p:---- s:0035 e:000034 CFUNC  :instance_eval
    c:0009 p:0039 s:0032 e:000031 METHOD /Users/rich/.rvm/gems/ruby-2.3.1/gems/rack-1.6.4/lib/rack/builder.rb:55 [FINISH]
    c:0008 p:---- s:0027 e:000026 CFUNC  :new
    c:0007 p:0013 s:0024 E:0008e8 EVAL   config.ru:1 [FINISH]
    c:0006 p:---- s:0022 e:000021 CFUNC  :eval
    c:0005 p:0150 s:0016 e:000015 METHOD /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:110
    c:0004 p:0115 s:0009 e:000008 CLASS  /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:156
    c:0003 p:0011 s:0006 e:000005 CLASS  /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:30
    c:0002 p:0036 s:0004 E:000448 EVAL   /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:29 [FINISH]
    c:0001 p:0000 s:0002 E:000398 TOP    [FINISH]

    /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:29:in `<main>'
    /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:30:in `<module:PhusionPassenger>'
    /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:156:in `<module:App>'
    /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:110:in `preload_app'
    /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb:110:in `eval'
    config.ru:1:in `<main>'
    config.ru:1:in `new'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `initialize'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `instance_eval'
    config.ru:5:in `block in <main>'
    config.ru:5:in `require'
    /Library/WebServer/Documents/HQ/app.rb:7:in `<top (required)>'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler.rb:106:in `require'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:75:in `require'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:75:in `each'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:86:in `block in require'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:86:in `each'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/runtime.rb:91:in `require'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/pg-0.19.0/lib/pg.rb:4:in `<top (required)>'
    /Users/rich/.rvm/gems/ruby-2.3.1/gems/pg-0.19.0/lib/pg.rb:4:in `require'

    -- C level backtrace information -------------------------------------------

    -- Other runtime information -----------------------------------------------

    * Loaded script: /usr/local/Cellar/passenger/5.0.30_2/libexec/src/helper-scripts/rack-preloader.rb

    * Loaded features:

            0 enumerator.so
            1 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/enc/encdb.bundle
            2 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/enc/trans/transdb.bundle
            3 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/rbconfig.rb
            4 /Library/Ruby/Site/2.0.0/rubygems/compatibility.rb
            5 /Library/Ruby/Site/2.0.0/rubygems/defaults.rb
            6 /Library/Ruby/Site/2.0.0/rubygems/deprecate.rb
            7 /Library/Ruby/Site/2.0.0/rubygems/errors.rb
            8 /Library/Ruby/Site/2.0.0/rubygems/version.rb
            9 /Library/Ruby/Site/2.0.0/rubygems/requirement.rb
         10 /Library/Ruby/Site/2.0.0/rubygems/platform.rb
         11 /Library/Ruby/Site/2.0.0/rubygems/basic_specification.rb
         12 /Library/Ruby/Site/2.0.0/rubygems/stub_specification.rb
         13 /Library/Ruby/Site/2.0.0/rubygems/util/stringio.rb
         14 /Library/Ruby/Site/2.0.0/rubygems/specification.rb
         15 /Library/Ruby/Site/2.0.0/rubygems/exceptions.rb
         16 /Library/Ruby/Site/2.0.0/rubygems/core_ext/kernel_gem.rb
         17 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/thread.rb
         18 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/monitor.rb
         19 /Library/Ruby/Site/2.0.0/rubygems/core_ext/kernel_require.rb
         20 /Library/Ruby/Site/2.0.0/rubygems.rb
         21 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger.rb
         22 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/constants.rb
         23 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/etc.bundle
         24 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/fileutils.rb
         25 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/tmpdir.rb
         26 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils/tmpio.rb
         27 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/platform_info.rb
         28 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/platform_info/operating_system.rb
         29 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/platform_info/ruby.rb
         30 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/platform_info/binary_compatibility.rb
         31 /usr/local/Cellar/passenger/5.0.30_2/libexec/buildout/ruby/ruby-2.0.0-universal-macosx/passenger_native_support.bundle
         32 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/native_support.rb
         33 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/socket.bundle
         34 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/socket.rb
         35 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/pathname.bundle
         36 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/pathname.rb
         37 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/ruby_core_enhancements.rb
         38 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/ruby_core_io_enhancements.rb
         39 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/base64.rb
         40 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils.rb
         41 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/preloader_shared_helpers.rb
         42 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/public_api.rb
         43 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/debug_logging.rb
         44 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/shellwords.rb
         45 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils/shellwords.rb
         46 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb
         47 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/fcntl.bundle
         48 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/message_channel.rb
         49 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/message_client.rb
         50 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils/native_support_utils.rb
         51 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils/unseekable_socket.rb
         52 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/request_handler/thread_handler.rb
         53 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/request_handler.rb
         54 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/stringio.bundle
         55 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/utils/tee_input.rb
         56 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/rack/thread_handler_extension.rb
         57 /Library/Ruby/Site/2.0.0/rubygems/path_support.rb
         58 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/vendor/union_station_hooks_core/lib/union_station_hooks_core/version.rb
         59 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/vendor/union_station_hooks_core/lib/union_station_hooks_core.rb
         60 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/vendor/union_station_hooks_rails/lib/union_station_hooks_rails/version.rb
         61 /usr/local/Cellar/passenger/5.0.30_2/libexec/src/ruby_supportlib/phusion_passenger/vendor/union_station_hooks_rails/lib/union_station_hooks_rails.rb
         62 /Library/Ruby/Site/2.0.0/rubygems/dependency.rb
         63 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/postit_trampoline.rb
         64 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/constants.rb
         65 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/io/console.bundle
         66 /Library/Ruby/Site/2.0.0/rubygems/user_interaction.rb
         67 /Library/Ruby/Site/2.0.0/rubygems/config_file.rb
         68 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/rubygems_integration.rb
         69 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/current_ruby.rb
         70 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/shared_helpers.rb
         71 /Library/Ruby/Site/2.0.0/rubygems/ext/build_error.rb
         72 /Library/Ruby/Site/2.0.0/rubygems/ext/builder.rb
         73 /Library/Ruby/Site/2.0.0/rubygems/ext/configure_builder.rb
         74 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/delegate.rb
         75 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/tempfile.rb
         76 /Library/Ruby/Site/2.0.0/rubygems/ext/ext_conf_builder.rb
         77 /Library/Ruby/Site/2.0.0/rubygems/ext/rake_builder.rb
         78 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/optparse.rb
         79 /Library/Ruby/Site/2.0.0/rubygems/command.rb
         80 /Library/Ruby/Site/2.0.0/rubygems/ext/cmake_builder.rb
         81 /Library/Ruby/Site/2.0.0/rubygems/ext.rb
         82 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/errors.rb
         83 /Users/rich/.rvm/gems/ruby-2.3.1/gems/bundler-1.13.6/lib/bundler/environment_preserver.rb
         84 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb
         85 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb
         86 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/common.rb
         87 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/generic.rb
         88 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/ftp.rb
         89 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/http.rb
         90 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/https.rb
         91 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/ldap.rb
         92 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/ldaps.rb
         93 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri/mailto.rb
         94 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/uri.rb
         95 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin15/zlib.bundle
         96 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/exceptions.rb
         97 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/header.rb
         98 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/generic_request.rb
         99 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/request.rb
        100 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/requests.rb

Answer 1

Turns out you can't point directly to a hard binary. RVM needs that pointer to a wrapper, so this is the final path: passenger_ruby /Users/rich/.rvm/wrappers/default/ruby, which is a pointer to passenger_ruby /Users/rich/.rvm/wrappers/ruby-2.3.1/ruby. Tested both applications and it works completely now. The command brew info nginx --with-passenger output the wrong ruby location.

Answer 2

It looks like your pg gem compiled against 2.0.0p648 when you originally installed it. Simply running

gem uninstall pg

rvm use ruby_2_3_1 (switch to Ruby 2.3.1)

gem install pg (This should recompile it against Ruby 2.3.1)

Also specify ruby version in passenger config file.

passenger_ruby /Users/rich/.rvm/gems/ruby-2.3.1/bin/ruby;