CODEWITHSUNDEEP
mongodb
Jonesome Reinstate Monica
Jonesome Reinstate Monica

Reputation: 6934

Mongodb: Does not seem to respect capped size limit

Running mongodb on win2012

Mongo db version v2.6.4

Our collection is created with this param:

capped:true, size:500000000000 // 500 gb

But the data dir, that only holds this one collection, has grown to 720gb.

There is no replication in use at all on this mongo server.

What can I do to keep the data dir at ~ 500 gb?

This db does get massive amounts of inserts at times, but we also need to control the max size (and let it discard old data).

Is it possible that the capped size does not include indexes?

Or is there other overhead I should account for? (and how?

Or?

db stats

> db.stats
function (scale){
    return this.runCommand( { dbstats : 1 , scale : scale } );
}
> db.stats()
{
        "db" : "logging",
        "collections" : 3,
        "objects" : 289637086,
        "avgObjSize" : 2076.3166847770317,
        "dataSize" : 601378314192,
        "storageSize" : 606012620432,
        "numExtents" : 305,
        "indexes" : 12,
        "indexSize" : 147827443456,
        "fileSize" : 776943435776,
        "nsSizeMB" : 16,
        "dataFileVersion" : {
                "major" : 4,
                "minor" : 5
        },
        "extentFreeList" : {
                "num" : 0,
                "totalSize" : 0
        },
        "ok" : 1
}
>

The command we use to create the collection:

db.createCollection("LogItem", { capped:true, size:500000000000 })

So the collection we write to is created with the capped cmd. and that is where all the space is

I can't do db.collection.stats() because out of disk space...

Thank you!

Upvotes: 2

Views: 917

Answers (2)

Jonesome Reinstate Monica
Jonesome Reinstate Monica

Reputation: 6934

RESOLVED

The issue? Well, you can create a collection as capped, and that DOES work.... but when creating the new db, if incoming connections are sending data, the new db is auto created before you get a chance to create it as capped, and you end up with... an uncapped collection.

Solution:

When mongo is taken off line, bring it back up temporarily with a config setting it to a non standard port. Create the collection. Then restart mongo on normal port, and all is well.

Upvotes: 3

alernerdev
alernerdev

Reputation: 2064

I believe you have lumped together multiple distinct concepts into one. The notion of "capped" applies to individual collections -- which reside in a database. A database is a file on disk consisting of 1 or more collections. Each database file has data plus some space to grow plus some holes from deleted data, etc. Capped collections do not have indexes -- so for retrieval, if you really are trying to create a 500gb collection, the performance is going to be horrendous. For testing, you can make a capped collection of 1mb in size, fill it up with data, and verify that once it reaches a certain number of documents, it does not grow -- but it doesnt mean that the database containing this collection will not exceed the desired size -- there is going to be some overhead.

Upvotes: 1

Related Questions