CODEWITHSUNDEEP
mongodbdatabase-performancespring-data-mongodb
akinKaplanoglu
akinKaplanoglu

Reputation: 768

16 MB size of aggregration pipeline in MongoDB

This is about a recommendation on mongodb. I have a collection that always increase row by row (I mean the count of documents). It is about 5 billion now. When I make a request on this collection I sometimes get the error about 16 MB size.

The first thing that I want to ask is which structure is the best way of creating collections that increasing the rows hugely. What is the best approach? What should I do for this kind of structure and the performance?

Upvotes: 1

Views: 962

Answers (1)

Juan Carlos Farah
Juan Carlos Farah

Reputation: 3879

Just to clarify, the 16MB limitation is on documents, not collections. Specifically, its the maximum BSON document size, as specified in this page in the documentation.

The maximum BSON document size is 16 megabytes.

If you're running into the 16MB limit in aggregation is because you are using MongoDB version 2.4 or older. In these, the aggregate() method returned a document, which is subject to the same limitation as all other documents. Starting in 2.6, the aggregate() method returns a cursor, which is not subject to the 16MB limit. For more information, you should consult this page in the documentation. Note that each stage in the aggregation pipeline is still limited to 100MB of RAM.

Upvotes: 1

Related Questions