CODEWITHSUNDEEP
mongodblarge-data
Arthur
Arthur

Reputation: 5148

MongoDB with LOTS OF datas?

I'm a beginner with a non SQL structure like here with MongoDB and I don't find somebody talk about a collection with lots of data, like 1.000.000 entries ? and more ?

I saw a company page on the official site. But nothing with large data companies.

I heard about a combo with SQL : Large data are stocked on SQL tables, and only the "cache" are on MongoDB, but it's the only one solution for MongoDB and large data ?

Upvotes: 1

Views: 169

Answers (2)

preinheimer
preinheimer

Reputation: 3722

We're using MongoDB to power Where's it Up, and the api behind it. We're currently pushing in >3 million documents per day. MongoDB is the only storage engine in use. We were keeping a bunch around for a while, but we're now using TTL to delete old records.

Things are going super well, just make sure you have all the indexes you need. Querying a million+ records without an index is bad, regardless of your storage engine. Auto-failover has been super helpful.

Something to watch out for is updating records to include more information, it can be pretty expensive if the document grows past pre-allocated space. We ended up changing how we stored data to avoid updates, and create new documents instead.

Upvotes: 3

Markus W Mahlberg
Markus W Mahlberg

Reputation: 20703

MongoDB in it's current incarnation is explicitly designed to make it easy to scale out.

As for the numbers: one of my test databases has 10M records and runs easily on my MacBook Air, which is 4 years old now.

So what you can do when your current cluster can not handle the data stored (either because the indices are too big for your RAM or because of processing the queries takes too long): add another node to your MongoDB cluster. Your performance gain should be something between slightly below linear (if your cluster was in perfect condition otherwise) up to several orders of magnitude (when indices didn't fit into RAM and/or IO was pushed to it's limits before and that situation changed after scaling out).

A word of warning: you should have somebody who knows about MongoDB administration in case you want to put you deployment into production. Though MongoDB administration seems to be easy, it is by no means something to be done by a layman. Especially not for production use.

Upvotes: 1

Related Questions