PG::Error: ERROR: relation "users" does not exist on Heroku

Question

I'm experiencing a very odd error, only intermittently. I just completed an upgrade from Rails 3.0 to Rails 3.2. The 3.0 app has been running on Heroku's Bamboo stack, and concurrent to the upgrade, I'm also changing to the more modern Cedar stack.

Heroku's instructions for migrating say to use PG Backups to migrate data from the old Bamboo app to the new Cedar app.

I've done everything according to the instructions given in the documents above. Now, my app mostly works. I've tested all the features, checked/loaded all the pages, and the new Cedar app is running mostly fine.

Here's the reason for the mostly: Intermittently, instead of loading a page that was working just seconds ago, the app will give a 500 error. The error is ActiveRecord::StatementInvalid: PG::Error: ERROR: relation "users" does not exist and if you refresh the same page a few times, it goes away and the page loads. Keep refreshing, and it will eventually give the error again.

Everything I've read about the error says to run rake db:migrate, but I've already done that on the old Bamboo app before doing the data migration! There were no database changes involved in the upgrade from 3.0 to 3.2. Also, if it were a migration issue, the page shouldn't ever load.

I've included the backtrace from Airbrake below. Help is appreciated.

vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:1174:in `get_last_result'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:1174:in `exec_cache'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:661:in `block in exec_query'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/active_record.rb:46:in `block in log_with_newrelic_instrumentation'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/method_tracer.rb:273:in `trace_execution_scoped'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/active_record.rb:43:in `log_with_newrelic_instrumentation'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:659:in `exec_query'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/postgresql_adapter.rb:1256:in `select'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/database_statements.rb:18:in `select_all'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/query_cache.rb:61:in `block in select_all'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/query_cache.rb:75:in `cache_sql'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/query_cache.rb:61:in `select_all'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/querying.rb:38:in `block in find_by_sql'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/explain.rb:41:in `logging_query_plan'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/querying.rb:37:in `find_by_sql'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/method_tracer.rb:523:in `block in find_by_sql_with_trace_ActiveRecord_self_name_find_by_sql'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/method_tracer.rb:273:in `trace_execution_scoped'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/method_tracer.rb:518:in `find_by_sql_with_trace_ActiveRecord_self_name_find_by_sql'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation.rb:171:in `exec_queries'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation.rb:160:in `block in to_a'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/explain.rb:41:in `logging_query_plan'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation.rb:159:in `to_a'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/finder_methods.rb:380:in `find_first'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/finder_methods.rb:122:in `first'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/finder_methods.rb:338:in `find_one'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/finder_methods.rb:314:in `find_with_ids'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/finder_methods.rb:107:in `find'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/querying.rb:5:in `find'
app/controllers/users_controller.rb:18:in `show'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/abstract_controller/base.rb:167:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/rendering.rb:10:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:425:in `_run__1799501095522236001__process_action__1043849600952898910__callbacks'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:405:in `__run_callback'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:81:in `run_callbacks'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/abstract_controller/callbacks.rb:17:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/rescue.rb:29:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `block in instrument'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `instrument'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:38:in `block in process_action'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:318:in `perform_action_with_newrelic_trace'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:37:in `process_action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/abstract_controller/base.rb:121:in `process'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/abstract_controller/rendering.rb:45:in `process'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal.rb:203:in `dispatch'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_controller/metal.rb:246:in `block in action'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/routing/route_set.rb:73:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/routing/route_set.rb:36:in `call'
vendor/bundle/ruby/1.9.1/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
vendor/bundle/ruby/1.9.1/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
vendor/bundle/ruby/1.9.1/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/routing/route_set.rb:612:in `call'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/rack/error_collector.rb:12:in `call'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/rack/agent_hooks.rb:18:in `call'
vendor/bundle/ruby/1.9.1/gems/warden-1.2.1/lib/warden/manager.rb:35:in `block in call'
vendor/bundle/ruby/1.9.1/gems/warden-1.2.1/lib/warden/manager.rb:34:in `catch'
vendor/bundle/ruby/1.9.1/gems/warden-1.2.1/lib/warden/manager.rb:34:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/etag.rb:23:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/conditionalget.rb:25:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/head.rb:14:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/flash.rb:242:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/cookies.rb:341:in `call'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/query_cache.rb:64:in `call'
vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:405:in `_run__2580604918081661655__call__521120290706034756__callbacks'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:405:in `__run_callback'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/callbacks.rb:81:in `run_callbacks'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/sendfile.rb:102:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/rack/logger.rb:32:in `call_app'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/rack/logger.rb:16:in `block in call'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/tagged_logging.rb:22:in `tagged'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/rack/logger.rb:16:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/request_id.rb:22:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.13/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.13/lib/action_dispatch/middleware/static.rb:63:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
vendor/bundle/ruby/1.9.1/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:16:in `block in call'
/usr/local/lib/ruby/1.9.1/timeout.rb:58:in `timeout'
vendor/bundle/ruby/1.9.1/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:16:in `call'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/engine.rb:479:in `call'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/application.rb:223:in `call'
vendor/bundle/ruby/1.9.1/gems/railties-3.2.13/lib/rails/railtie/configurable.rb:30:in `method_missing'
vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:552:in `process_client'
vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:632:in `worker_loop'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22:in `call'
vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.2.96/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22:in `block (4 levels) in <top (required)>'
vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:500:in `spawn_missing_workers'
vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:142:in `start'
vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.2/bin/unicorn:126:in `<top (required)>'
vendor/bundle/ruby/1.9.1/bin/unicorn:19:in `load'
vendor/bundle/ruby/1.9.1/bin/unicorn:19:in `<main>'

Answer 1

Ok, you ran db:migrate before the migration - but do it again.

heroku run rake db:migrate

You can also restart the app

heroku restart

Is the app running on a development machine as well? If no version numbers for the gems is specified in the gemfile then Heroku will use the latest. Check if both Heroku and your dev machine is running the same versions or better specify it in the gemfile.