CODEWITHSUNDEEP
distributeddistributed-systemconsistencyconsensusraft
Sid
Sid

Reputation: 135

Raft loss of data/Log entries on leader re-election

Scenario 1: Lets assume there is a 5 node cluster, out of which 3 are already out of commission. There are 3 nodes in service A(leader), B, C

A replicated a log entry to B, C, received successful response, committed it, applied it, responded to the client and then died. Now, there are only two nodes B, C with some un-applied log entries. Now, if D comes up, and B becomes the new leader, what will happen to the un-applied entries ? will they be committed/applied as well ?

Scenario 2: Out of 5 node cluster, 3 just got down, A is still the leader and B is online

A replicated an entry to B, but cannot commit, and then A gets killed, C, D come up (so B, C, D are up). What will happen to the entries that were replicated to B? will they be committed/applied ?

Upvotes: 1

Views: 626

Answers (1)

kuujo
kuujo

Reputation: 8185

Keep in mind there's a difference between an entry being committed and an entry being applied. Entries can be committed but not yet be applied, but entries can't be applied and not committed. Once an entry is committed (replicated to a majority of the cluster in the leader's current term) it's guaranteed to eventually be applied on all nodes. So, the next leader must apply it, otherwise it could result in an inconsistent state if you're using Raft to manage a replicated state machine. The election protocol guarantees that the next leader elected will have the committed entries (so either B or C, but not D), and the new leader's first action will be to replicate those entries to another node (D), determine they're committed, then apply and update followers.

In the second case, if B gets elected with the additional entries then it will ultimately commit and apply them. This is why sessions are a critical component of guaranteeing linearizability for clients. When an operation fails due to failure to reach a quorum, it's always possible that change could actually be later committed. Sessions should be used to guarantee that a change is only applied to the state machine once no matter how many times it's committed (idempotence). See the Raft dissertation for more on sessions.

Upvotes: 1

Related Questions