CODEWITHSUNDEEP
mongodb
Kwan
Kwan

Reputation: 257

mongodb TTL not working

I had executed this command to set a TTL Index on mongodb, db.sessions.ensureIndex({'expiration':1},{"expireAfterSeconds" : 30})

but after 4 days,I found these documents were not removed. I had confirmed command and document's field was correct.

I don't know how to fix it.

after executed db.serverStatus(), I got localTime is 2015-01-16 11:03:05.554+08:00

and the following is some info of my collection

db.sessions.getIndexes()

{
"0" : {
    "v" : 1,
    "key" : {
        "_id" : 1
    },
    "name" : "_id_",
    "ns" : "meta.sessions"
},
"1" : {
    "v" : 1,
    "key" : {
        "expiration" : 1
    },
    "name" : "expiration_1",
    "ns" : "meta.sessions",
    **"expireAfterSeconds" : 30**
}
}

db.sessions.find()

/* 0 */
{
    "_id" : ObjectId("54b4c2e0f840238ca1436788"),
    "data" : ...,
    "expiration" : **ISODate("2015-01-13T16:02:33.947+08:00"),**
    "sid" : "..."
}

/* 1 */
{
    "_id" : ObjectId("54b4c333f840238ca1436789"),
    "data" : ...,
    "expiration" : ISODate("2015-01-13T16:06:56.942+08:00"),
    "sid" : ".."
}
/* ... */

Upvotes: 9

Views: 21477

Answers (3)

Sebastian
Sebastian

Reputation: 5845

This is an old question, I just wanted to share my findings and summarize on this because I had the same issue as the OP.

More info on TTL indexes and its troubleshootings:

  • This works as mentioned above by having a mongod thread running every 60 seconds to check which documents to remove by the identified index.
  • That thread is referred as the TTL Monitor
  • You can diagnose the TTL Monitor by running:
    • db.serverStats().metrics.ttl. passes determines the amount of times the TTL Monitor has run. So if you never disabled the TTL Monitor and your mongo db uptime is 2 minutes, you see 2 passes there.
  • By getting the ttlMonitorEnabled parameter as mentioned above:
    • db.adminCommand({ getParameter:1, ttlMonitorEnabled: 1 }). If ttlMonitoreEnabled equals true, then it's enabled.
    • if it's false, then you should enabled it:
      • db.adminCommand({ setParameter: 1, ttlMonitorEnabled: true })
  • You can check log files searching for ttl, in my case, using docker would be:
    • docker logs mongo-container | grep -i ttl
      • in my case, it was disabled because the database was in an inconsistent state.

This was my issue actually, the TTL monitor was not running due to multiple reuse of the same docker-compose along the lifespan of my environment.

Document(s) exist in 'system.replset', but started without --replSet.
Database contents may appear inconsistent with the writes that were
visible when this node was running as part of a replica set. Restart
with --replSet unless you are doing maintenance and no other clients
are connected. The TTL collection monitor will not start because of this.
For more info see http://dochub.mongodb.org/core/ttlcollections

Upvotes: 1

jerogaren
jerogaren

Reputation: 344

To expire data from a collection (Tested in version 3.2) you must create indexes:

db.my_collection.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } )

After that, every document that you insert in this collection must have the "createdAt" with the current date:

    db.my_collection.insert( {
   "createdAt": new Date(),
   "dataExample": 2,
   "Message": "Success!"
} )

The document will be removed when the date createdAt value + expireAfterSeconds value arrive. Note: This background task in MongoDB, by default, happens once every 60 seconds.

Upvotes: 14

jbochniak
jbochniak

Reputation: 1856

When you create TTL index in the foreground (like you did), MongoDB begins removing expired documents as soon as the index finishes building. Best to tail -f mongod.log during index creation to track the progress. You may wish to remove & recreate index if something went wrong.

If index was created in the background, the TTL thread can begin deleting documents while the index is building.

TTL thread that removes expired documents runs every 60 seconds.

If you created index on the replica that was taken out of the replica set and is running in standalone mode index WILL be created but documents will NOT be removed until you rejoin (or remove replica set) configuration. If this is the case you may get something similar to this in the mongod.log

** WARNING: mongod started without --replSet yet 1 documents are ** present in local.system.replset ** Restart with --replSet unless you are doing maintenance and no other ** clients are connected. ** The TTL collection monitor will not start because of this. ** For more info see http://dochub.mongodb.org/core/ttlcollections

Upvotes: 10

Related Questions