CODEWITHSUNDEEP
mongodbmongodb-query
Robins Gupta
Robins Gupta

Reputation: 3153

Using nested document structure in mongodb

I am planning to use a nested document structure for my MongoDB Schema design as I don't want to go for flat schema design as In my case I will need to fetch my result in one query only.

Since MongoDB has a size limit for a document. MongoDB Limits and Threshold

A MongoDB document has a size limit of 16MB ( an amount of data). If your subcollection can growth without limits go flat.

I don't need to fetch my nested data but only be needing my nested data for filtering and querying purpose.

I want to know whether I will still be bound by MongoDB size limits even if I use my embedded data only for querying and filter purpose and never for fetching of nested data because as per my understanding, in this case, MongoDB won't load the complete document in memory but only the selected fields?

Nested schema design example

{
    clinicName: "XYZ Hopital",
    clinicAddress: "ABC place.",
    "doctorsWorking":{
        "doctorId1":{
            "doctorJoined": ISODate("2017-03-15T10:47:47.647Z")
        },
        "doctorId2":{
            "doctorJoined": ISODate("2017-04-15T10:47:47.647Z")
        },
        "doctorId3":{
            "doctorJoined": ISODate("2017-05-15T10:47:47.647Z")
        },
        ...
        ...
        //upto 30000-40000 more records suppose
    }
}

Upvotes: 1

Views: 1724

Answers (1)

love gupta
love gupta

Reputation: 529

I don't think your understanding is correct when you say "because as per my understanding, in this case, MongoDB won't load the complete document in memory but only the selected fields?".

If we see MongoDB Doc. then it reads

The maximum BSON document size is 16 megabytes. The maximum document size helps ensure that a single document cannot use excessive amount of RAM or, during transmission, excessive amount of bandwidth. To store documents larger than the maximum size, MongoDB provides the GridFS API.

So the clear limit is 16 MB on document size. Mongo should stop you from saving such a document which is greater than this size.

If I agree with your understanding for a while then let's say that it allows to save any size of document but more than 16 MB in RAM is not allowed. But on other hand, while storing the data it won't know what queries will be run on this data. So ultimately you will be inserting such big documents which can't be used later. (because while inserting we don't tell the query pattern, we can even try to fetch the full document in a single shot later).

If the limit is on transmission (hypothetically assuming) then there are lot of ways (via code) software developers can bring data into RAM in clusters and they won't cross 16 MB limit ever (that's how they do IO ops. on large files). They will make fun of this limit and just leave it useless. I hope MongoDB creators knew it and didn't want it to happen.

Also if limit is on transmission then there won't be any need of separate collection. We can put everything in a single collections and just write smart queries and can fetch data. If fetched data is crossing 16 MB then fetch it in parts and forget the limit. But it doesn't go this way.

So the limit must be on document size else it can create so many issues.

In my opinion if you just need "doctorsWorking" data for filtering or querying purpose (and if you also think that "doctorsWorking" will cause document to cross 16 MB limit) then it's good to keep it in a separate collection.

Ultimately all things depend on query and data pattern. If a doctor can serve in multiple hospitals in shifts then it will be great to keep doctors in separate collection.

Upvotes: 2

Related Questions