CODEWITHSUNDEEP
ruby-on-railsmongodbindexingmongoidnosql
RaceCondition
RaceCondition

Reputation: 317

When and how often to build indexes with MongoDB in production

I am about to deploy a web app using MongoDB as storage. The problem is, that locally I can build the indexes whenever I want (also done automatically) - but you shouldn't do that in production. Now switching to production I am not quite sure on when to build the indexes and how often. Should I do it every day? Should I do it once a week? My project is like a small social network - so let's say I will have about 100.000 users someday. Depending on the amount of data it could take hours or even days to build indexes.

I intend to build my indexes in the background (see here http://www.mongodb.org/display/DOCS/Indexing+as+a+Background+Operation). It will still cause a write lock, but the build job will pause to allow other readers and writers to access the database - so if I there would be a heavy load on the database, a background index build would still affect the performance in a negative way. So it should be done within a time window where the traffic is at a minimum (researched here http://www.javabeat.net/articles/353-indexing-in-practice-using-mongodb-1.html).

Does anybody have experience on when and how often to create indexes in production (preferably with a large amount of data)? I am using Rails with Mongoid (http://mongoid.org/).

Upvotes: 3

Views: 1054

Answers (1)

Remon van Vliet
Remon van Vliet

Reputation: 18615

You seem to assume you have to periodically rebuild indexes. This is not the case. MongoDB updates indexes as you write to the related data.

The only possible reason I can see for production index rebuilds are :

  • Live optimization (e.g. adding an index to a field because it will dramatically improve performance)
  • After deploys that involve schema changes
  • After index related errors (should not occur)

Upvotes: 2

Related Questions