CODEWITHSUNDEEP
mongodbmorphia
illawara strathfield
illawara strathfield

Reputation: 1

MongoDB multiple scalable collections

I'm using morphia to connect to mongoDB. I'm collecting daily mileage for cars. Right now, all daily mileage for all cars are stored in 1 collection with the following attribute: plateNumber, date, mileage

we want to store the daily mileages from all the way back in 1990 onwards. Right now, we're already maintaining around 4500+ cars (that's roughly 1.3 mil records a year). We're trying with one year worth of data, and the performance is already slagging really badly. I was thinking of splitting the storage into multiple collections based on the plate number. so each plate number will have its own collection named after the plate number. I need some ideas. Is there any other way to solve this?

Adding details: How we'll use the data: we want to query mileages of multiple cars (sometimes per department, or per geographic area, per make/model, etc) at any given date range. So, lets just say we want to monitor mileages in a suburb, we'll take all plate numbers' mileages operating in that suburb from 01 Jan 2014 to 23 Jun 2014 and perform calculation on the data.

thanks.

Upvotes: 0

Views: 91

Answers (1)

Pio
Pio

Reputation: 4062

Depending on what is your configuration you can try Sharding or you may attempt to Partition your db -- though this approach is hybrid, meaning that you would mimic partitioning from sql database systems (Oracle, Sql Server, etc.).

Also note that if you insert (basically append) a lot of entries to a single file it will gradually become slow since mongo needs to update the primary key (mongoID) that needs to be unique + if you defined other indexes on the collection those also need to be updated.

If you can provide more information on how you intend to use the collected data and in what time intervals + are these operations online or offline I'll update my answer.

Upvotes: 1

Related Questions