mongodb replication node stuck at "STARTUP2" with optimeDate as 1970

Question

i have just setup replica sets with three nodes . the third node is stuck at stateStr STARTUP2 with "optimeDate" : ISODate("1970-01-01T00:00:00Z"). However its showing no error message. Is this alright. On primary rs.status() yeilds

{
    "set" : "qdit",
    "date" : ISODate("2013-06-18T22:49:41Z"),
    "myState" : 1,
    "members" : [
        {
            "_id" : 0,
            "name" : "q.example.com:27017",
            "health" : 1,
            "state" : 1,
            "stateStr" : "PRIMARY",
            "uptime" : 2940,
            "optime" : {
                "t" : 1371593311,
                "i" : 1
            },
            "optimeDate" : ISODate("2013-06-18T22:08:31Z"),
            "self" : true
        },
        {
            "_id" : 1,
            "name" : "q1.example.com:27017",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 457,
            "optime" : {
                "t" : 1371593311,
                "i" : 1
            },
            "optimeDate" : ISODate("2013-06-18T22:08:31Z"),
            "lastHeartbeat" : ISODate("2013-06-18T22:49:40Z"),
            "lastHeartbeatRecv" : ISODate("2013-06-18T22:49:40Z"),
            "pingMs" : 0,
            "syncingTo" : "twitnot.es:27017"
        },
        {
            "_id" : 2,
            "name" : "q2.example.com:27017",
            "health" : 1,
            "state" : 5,
            "stateStr" : "STARTUP2",
            "uptime" : 300,
            "optime" : {
                "t" : 0,
                "i" : 0
            },
            "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
            "lastHeartbeat" : ISODate("2013-06-18T22:49:40Z"),
            "lastHeartbeatRecv" : ISODate("2013-06-18T22:49:41Z"),
            "pingMs" : 7
        }
    ],
    "ok" : 1
}

also

db.printSlaveReplicationInfo() on yields

source:   qdit1.queuedit.com:27017
     syncedTo: Tue Jun 18 2013 22:08:31 GMT+0000 (UTC)
         = 2894 secs ago (0.8hrs)
source:   qdit2.queuedit.com:27017
     syncedTo: Thu Jan 01 1970 00:00:00 GMT+0000 (UTC)
         = 1371596205 secs ago (380998.95hrs)

Is this alright. Also how can i test my replication especially the third node

Answer 1

Your STARTUP2 status should get resolved as soon as the replica set is in full sync. This would typically happen when the replica had lag behind due to hardware or any other problem and you either restart the mongod instance or reboot the node. Note that the STARTUP phase is completely different than the STARTUP2 phase where it has completed parsing of configuration and is forking threads as well doing the initial sync, in the later phase. This phase will not resolve only if the oplog first event time as seen in db.printReplicationInfo() is later than the time you see in your db.printSlaveReplicationInfo() command for that respective slave node, in which case a full manual resync will be necessary. So if this time is within the first and last event time of the oplog in the PRIMARY instance, the STARTUP2 phase should only be transient.