CODEWITHSUNDEEP
ruby-on-railsrubymongodbmongoid
JP Silvashy
JP Silvashy

Reputation: 48535

MongoDB redundant queries on system.namespaces collection?

So I've been using MongoDB (with the Mongoid Ruby Gem) for a while now, and as our app has grown I've noticed requests taking longer and longer as my data has grown, here is what a typical request for my app looks like, but it takes about 500ms, just for the DB stuff.

Nothing special here just some controller stuff:

Started GET "/cities/san-francisco?date_range=past_week" for 127.0.0.1 at 2011-11-15 11:13:04 -0800
  Processing by CitiesController#show as HTML
  Parameters: {"date_range"=>"past_week", "id"=>"san-francisco"}

Then the queries run, but what I don't understand is that for every query that runs it performs a MONGODB dashboard_development['system.namespaces'].find({}) before actually running it! Why?

MONGODB dashboard_development['system.namespaces'].find({})
MONGODB dashboard_development['users'].find({:_id=>BSON::ObjectId('4e80e0090f6d2e306f000001')})
MONGODB dashboard_development['system.namespaces'].find({})
MONGODB dashboard_development['cities'].find({:slug=>"san-francisco"})
MONGODB dashboard_development['system.namespaces'].find({})
MONGODB dashboard_development['accounts'].find({:_id=>BSON::ObjectId('4e80e0090f6d2e306f000002')})
MONGODB dashboard_development['system.namespaces'].find({})
MONGODB dashboard_development['neighborhoods'].find({"city_id"=>BSON::ObjectId('4e80e00a0f6d2e306f000005')})

Then the views get rendered, they are pretty slow too... but that is a seperate problem all together, I'll address that at another time.

Rendered cities/_title_and_scope.html.erb (109.3ms)
Rendered application/_dropdown.html.erb (0.1ms)
Rendered application/_date_range_selector.html.erb (6.2ms)
Rendered cities/show.html.erb within layouts/application (122.7ms)
Rendered application/_user_dropdown.html.erb (0.9ms)
Rendered application/_main_navigation.html.erb (5.8ms)

So minus the views the request took about 500ms, thats too long for a really simple query, additionally the app is going to grow and that time is going to grow as well. Also this example is faster than the requests usually take, sometimes it takes 1000ms or more!

Completed 200 OK in 628ms (Views: 144.9ms)

Additionally I wanted to ask what fields are most appropriate for indexes? Maybe this is my problem, as I'm not really using them at all. Any help understanding this would be really really appreciated. Thanks!

Upvotes: 2

Views: 288

Answers (1)

Jesse Wolgamott
Jesse Wolgamott

Reputation: 40277

You need to use indexes -- otherwise, your mongo queries are executing what is best described as a full table scan. It is loading the entirety of your collection's json documents into memory and then evaluating each one to determine if it should be including in the response.

Strings, Date, Numbers can all be used as index -- the trick is, have an index on each attribute you are doing a "where" on.

You can turn off table-scans in your mongo config to help find table scans and destroy them!

Upvotes: 2

Related Questions