CODEWITHSUNDEEP
databasecassandraconsistencyscylla
Aleksandr Kretov
Aleksandr Kretov

Reputation: 11

Problem with consistency on Scylla (Cassandra)

Please help me with the following problem:

There is a Scylla cluster (5 nodes). Kafka messages are processed in the application. Scylla stores entity by сomposite partition key (externalId, systemId). When processing a Kafka message, Scylla stores the entity processing data, after processing the entity goes to the next stage of processing in another microservice.

After processing, a notification is received for this entity At this stage a floating problem appears: an entity with outdated data is received from Scylla

enter image description here

What we tried to do:

Upvotes: 0

Views: 484

Answers (1)

Nadav Har'El
Nadav Har'El

Reputation: 13761

In the ScyllaDB (and Cassandra) consistency model, you control the consistency level of the write and the read separately. If you want to be sure that after an update completed, a select will read the updated data, you need to pick certain combinations of write and read consistency levels that will allow that. For example writing with QUORUM (when RF=3, this means 2 out of 3) and then reading again with QUORUM, ensures that the read reaches at least one up-to-date node so you read the up-to-date data.

That being said, writing with consistency level ALL - which you said you tried - should have also been enough, no matter which consistency-level is used for read: In CL=ALL writes, the write waits until every single replica was successfully updated. At that point, a read to any node will return the updated data - there is no node left with outdated data to return! So if CL=ALL didn't work for you, my guess is that something in your application is wrong. For example, maybe one of the following happend:

  1. You thought you set CL=ALL on writes in your application code, but it had no effect.
  2. Some writes failed and you didn't notice. A CL=ALL write that fails doesn't guarantee consistency of what it wrote.

Upvotes: 0

Related Questions