CODEWITHSUNDEEP
cassandracassandra-3.0scylla
Rupesh Mukherjee
Rupesh Mukherjee

Reputation: 75

Commitlog sync period

According to cassandra commit log sync period..data first goes to os buffer...then from os buffer, based on commit log sync period ,the buffer data gets synced to commit log file in disk..and the default sync period is 10 seconds..what if server crashes in that 10 secs.. data is lost ryt? But the client got a response as SUCCESS the moment data is written to commitlog buffer in os buffer and memtable..but eventually data is lost because system crashed within that 10 sec window...am i missing something?

Upvotes: 3

Views: 1345

Answers (2)

TomerSan
TomerSan

Reputation: 1578

(Disclaimer: I'm a ScyllaDB employee)

I think what you are missing is that the data is written to commitlog on disk and to the memtable simultaneously, and assuming you are using RF > 1 with CL > 1 (e.g. quorum) than even if a specific node crashed the other replicas will still have the data, which can later be repaired.

If you are using RF > 1 and CL = ONE, there's also a chance that if the node crashed before the replica was synced, than the data will be lost.

If the entire cluster goes down, or in case of a single node cluster than indeed you client can get the SUCCESS ACK back, yet the data will be lost.

You are welcome to check Scylla Architecture docs for better understanding:

Upvotes: 3

Duarte Nunes
Duarte Nunes

Reputation: 852

You are not missing anything. Databases like Cassandra and Scylla not only tradeoff consistency for availability under failures, but, like traditional databases like Postgres, also tradeoff durability for performance. You can change the commitlog_sync option to batch or decrease the commitlog_sync_period_in_ms; note that if you do this, it is best to have the commitlog stored in a different media than the data directory.

The reasoning behind this is that durability can be achieved through persistency but also through replication. A typical Cassandra/Scylla user will usually have RF = 3, and write with a consistency level of QUORUM, such that you'd need coordinated failures of multiple machines to actually lose data.

Upvotes: 10

Related Questions