Reputation: 1711

Why a member of mongodb keep RECOVERING?

I set up a replica set with three members and one of them is an arbiter.

One time I restart a member, the member keep RECOVERING for a long time and did not be SECONDARY again, even though the database was not large.

The status of replica set is like that:

rs:PRIMARY> rs.status()
{
        "set" : "rs",
        "date" : ISODate("2013-01-17T02:08:57Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 1,
                        "name" : "192.168.1.52:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 67968,
                        "optime" : Timestamp(1358388479000, 1),
                        "optimeDate" : ISODate("2013-01-17T02:07:59Z"),
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "192.168.1.50:29017",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 107,
                        "lastHeartbeat" : ISODate("2013-01-17T02:08:56Z"),
                        "pingMs" : 0
                },
                {
                        "_id" : 3,
                        "name" : "192.168.1.50:27017",
                        "health" : 1,
                        "state" : 3,
                        "stateStr" : "RECOVERING",
                        "uptime" : 58,
                        "optime" : Timestamp(1358246732000, 100),
                        "optimeDate" : ISODate("2013-01-15T10:45:32Z"),
                        "lastHeartbeat" : ISODate("2013-01-17T02:08:55Z"),
                        "pingMs" : 0,
                        "errmsg" : "still syncing, not yet to minValid optime 50f6472f:5d"
                }
        ],
        "ok" : 1
}

How should I fix this problem?

Upvotes: 7

Answers (4)

Syed Abuthaheer

Reputation: 41

I've fixed the issue by following the below procedure.

Step1:

rs.remove("10.x.x.x:27017")

Step 2:

Stop the mongodb server on the issue node systemctl stop mongodb.service

Step 3:

Create a new new folder on the dbpath mkdir /opt/mongodb/data/db1 Note : existing path was /opt/mongodb/data/db

Step 4:

Modify dbpath on /etc/mongod.conf or mongdb yaml file dbPath: /opt/mongodb/data/db1

Step 5:

Start the mongodb service systemctl start mongodb.service

Step 6:

Takebackup of the existing folder and remove it

mkdir /opt/mongodb/data/backup mv /opt/mongodb/data/db/* /opt/mongodb/data/backup tar -cvf /opt/mongodb/data/backup.tar.gz /opt/mongodb/data/backup rm -rf /opt/mongodb/data/db/

Upvotes: 4