Reputation: 9826
If all nodes in a 3-nodes Percona Cluster have shutdown (gracefully shutdown or crash), from this blog, it says that when the nodes could reach each other, the cluster could recover automatically. However, starting the nodes in such a situation seems a difficult task.
So is there a reliable and operable method to do cluster recovery in this situation?
Upvotes: 1
Views: 1870
Reputation: 100
Another thing you can try to check which is most advanced node run below command on every node and check which node has largest committed transaction value.
mysqld_safe --wsrep-recover
start First node which has max committed value then second and third
Upvotes: 0
Reputation: 781
Examine the grastate.dat
file on all 3 nodes. Which node has the highest sequence number? You should bootstrap that node. Wait for it to come online. Then start node2. It should IST from the bootstrap node. Then start node3.
Golden rule: You must always bootstrap the very first node of any cluster. Bootstrapping does not erase data; it only starts a new cluster.
Depending on the version, you may need to set safe_to_bootstrap
in the grastate file to 1 manually.
Upvotes: 2