CODEWITHSUNDEEP
mysqlrubyruby-datamapper
jgnagy
jgnagy

Reputation: 171

DataMapper DB pool in Ruby App

I'm writing a multi-threaded Ruby application that is generating and loading data via dm-sweatshop into a legacy MySQL database for use in a load test. Everything is working fine with one exception, I'm loading several million records, but I can't seem to increase the DB connection pool size to get things done faster. It seems to be stuck at 10 connections. I've tried increasing the thread pool size of the application itself, but watching MySQL, I can't seem to get more than 10 connections established.

I originally was just using URL for the DataMapper setup:

DataMapper.setup(:default, 'mysql://user:password@db-server/testing')

But I've moved to this to try to set a DB pool size:

DataMapper.setup(:default, {
  :adapter  => 'mysql',
  :pool     => 20,
  :host     => 'db-server',
  :database => 'testing',
  :username => 'user',
  :password => 'password'
})

How do I increase the pool size? I feel like I've poured over every RDoc and piece of documentation I can find on DataMapper, but I can't find how to do it. I would have switched to ActiveRecord to make this work at this point, but it doesn't support composite primary keys, which I need for this legacy DB.

For reference, I'm utilizing the 'thread/pool' gem for multi-threading and the app runs in Ruby 1.9.3. Here is how I'm using DataMapper in multiple threads:

pool = Thread::Pool.new(@config[:thread_pool_size])
...
10000.times { pool.process {Customer.gen} }
...
pool.shutdown
sleep 5
puts "DONE!"

Upvotes: 0

Views: 534

Answers (1)

tsauerwein
tsauerwein

Reputation: 6011

Considering that this questions is very old and that DataMapper is no longer maintained it is kind of obsolete, but for the sake of completeness: DataMapper is using a hard-coded pool-size of 8 and is ignoring your configuration.

See also: Datamapper & connection pool logic

Upvotes: 1

Related Questions