CODEWITHSUNDEEP
rubyactiverecorddatabase-connectionsinatrapuma
Elena
Elena

Reputation: 51

Sinatra, Puma, ActiveRecord: No connection pool with 'primary' found

I am building a service in Ruby 2.4.4, with Sinatra 2.0.5, ActiveRecord 5.2.2, Puma 3.12.0. (I'm not using rails.)

My code looks like this. I have an endpoint which opens a DB connection (to a Postgres DB) and runs some DB queries, like this:

POST '/endpoint' do
  # open a connection
  ActiveRecord::Base.establish_connection(@@db_configuration)
  # run some queries
  db_value = TableModel.find_by(xx: yy)
  return whatever
end

after do
  # after the endpoint finishes, close all open connections
  ActiveRecord::Base.clear_all_connections!
end

When I get two parallel requests to this endpoint, one of them fails with this error:

2019-01-12 00:22:07 - ActiveRecord::ConnectionNotEstablished - No connection pool with 'primary' found.:
    C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/activerecord-5.2.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:1009:in `retrieve_connection'
    C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/activerecord-5.2.2/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
    C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/activerecord-5.2.2/lib/active_record/connection_handling.rb:90:in `connection'
    C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/activerecord-5.2.2/lib/active_record/core.rb:207:in `find_by'
...

My discovery process went this way so far.

  1. I looked at the connection usage in Postgres, thinking I might leak connections - no, I didn't seem to.
  2. Just in case, I increased the connection pool to 16 (corresponding to 16 Puma threads) - didn't help.

  3. Then I looked into the ActiveRecord sources. Here I realized why 2) didn't help. The problem is not that I can't get a connection, but I can't get a connection pool (yes, yes, it says that in the exception). The @owner_to_pool map variable, from which a connection pool is obtained, stores the process_id as key, and as values - connection pools (actually, the value is also a map, where the key is a connection specification and the value, I presume, is an actual pool instance). In my case, I have only one connection spec to my only db.

    But Puma is a multithreaded webserver. It runs all requests in the same process but in different threads.

    Because of that, I think, the following happens:

    • The first request, starting in process_id=X, thread=Y, "checks out" the connection pool in establish_connection, based on process_id=X, - and "takes" it. Now it's not present in the @owner_to_pool.
    • The second request, starting in the same process_id=X, but different thread=Z, tries to do the same - but the connection pool for process_id=X is not present in owner_to_pool. So the second request doesn't get a connection pool and fails with that exception.
    • The first request finished successfully and puts the connection pool for process_id=X back in place by calling clear_all_connections.
    • Another request, starting after all that, and not having any parallel requests in parallel threads, will succeed, because it will pick up the connection pool and put it back again with no problems.

Although I am not sure I understand everything 100% correctly, but it seems to me that something like this happens.

Now, my question is: what do I do with all this? How do I make the multithreaded Puma webserver work correctly with ActiveRecord's connection pool?

Thanks a lot in advance!

This question seems similar, but unfortunately it doesn't have an answer, and I don't have enough reputation to comment on it and ask the author if they solved it.

Upvotes: 3

Views: 1230

Answers (1)

Elena
Elena

Reputation: 51

So, basically, I didn't realize I was establish_connection is creating a connection pool. (Yes, yes, I said so myself in the question. Still, didn't quite realize it.)

What I ended up doing, is this:

require ....

# create the connection pool with the required configuration - once; it'll belong to the process
ActiveRecord::Base.establish_connection(db_configuration)

at_exit {
  # close all connections on app exit
  ActiveRecord::Base.clear_all_connections!
}

class SomeClass < Sinatra::Base
  POST '/endpoint' do
    # run some queries - they'll automatically use a connection from the pool
    db_value = TableModel.find_by(xx: yy)
    return whatever
  end
end

Upvotes: 2

Related Questions