CODEWITHSUNDEEP
mongodbaggregation-framework
songjhh
songjhh

Reputation: 267

Can we always use allowDiskUse:true in MongoDB aggregation query

Aggregation pipeline stages have a limit of 100 MB. To handle large datasets, allowDiskUse enables writing to temporary files. But have this optional any defect? If haven't, why the default is false?

Upvotes: 4

Views: 6546

Answers (2)

Dave
Dave

Reputation: 63

The performance impact described in the answer from @prasad_ still holds true (hard-drive or storage will be slower than memory).

However, starting in MongoDB 6.0 the behavior of allowDiskUse has changed as it now defaults to true. That is, unless one explicitly disallows disk use with allowDiskUse(false) MongoDB resorts to disk use for (non-index) sorts requiring more than 100 MB memory.

Upvotes: 2

prasad_
prasad_

Reputation: 14317

Unless you get an out-of-memory error during aggregation stages like sort, you don't need to use this option.

Note that working with data in memory is very much faster than working with data on hard drive. With hard drive data transfer the aggregation operation has to keep transferring data between the operation, memory and the hard disk.

So, by default allowDiskUse option is disabled (or set tofalse). You need to use this option only when needed - e.g., your sort operation needs more than 100 MB RAM and there is an error. Then you will have to use allowDiskUse:true to do your aggregation without errors. In such a case, note that when this option is used (set to true) the aggreagation operation will be slower.

If your aggregation is working fine and using this option allowDiskUse:true has no effect as the operation is able use the memory effectively (and the aggregation ignores this option).

Upvotes: 1

Related Questions