CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbconsistencyeventual-consistency
Mehran
Mehran

Reputation: 16841

Are DynamoDB Updates strongly consistent?

The whole reason why DynamoDB is fast and scalable is based on the fact that it is eventually consistent. But at the same time, it comes with this ConsistentRead option for operations like get, batchGet, and query which helps you make sure that the data you are reading is the latest one.

My question is about the update operation. First of all, it does not have the ConsistentRead option (one reason would be, update is not a read!). But at the same time, you can update a record in an atomic manner with ConditionExpression, like this:

await docClient.update({
    TableName: 'SomeTable',
    Key: {id},
    UpdateExpression: "set #status = :new_status",
    ConditionExpression: '#status = :old_status',
    ExpressionAttributeNames: {
        "#status": "status",
    },
    ExpressionAttributeValues: {
        ":old_status": "available",
        ":new_status": "done",
    },
}).promise()

This will make sure that at the time of update, the old value is available and if it isn't, the operation will fail with an exception thrown. So, in a sense, you can say that update is strongly consistent.

But my question is about a scenario in which you need to make sure the record exists. Let's say that you have one function which inserts a record. And another one that updates the same record (given its id). My concern is what if by the time the update operation is executed, because of eventually consistency of DynamoDB, there's no record matched and the update fails. As said before, the update operation does not come with a ConsistentRead option to make it strongly consistent.

Is this a valid concern? Is there anything I can do to help this?

Upvotes: 26

Views: 11454

Answers (3)

stanleywxc
stanleywxc

Reputation: 26

All of the answers above are incorrect. There is no write strong consistency.

UpdateItem will return when the leader node of the replication group is acknowledged once a quorum of peers persists the log record to their local write-ahead logs. This means that ONLY the write-ahead log is being replicated to the log node, not the actual storage node. Therefore UpdateItem doesn't guarantee successful writes in the replication group to all storage nodes.

Upvotes: -2

Jorge Garcia
Jorge Garcia

Reputation: 2570

There are no strongly consistent updates; strong consistency applies to reads where basically data viewed immediately after a write will be consistent for all observers of the entity.

When your application writes data to a DynamoDB table and receives an HTTP 200 response (OK), the write has occurred (in at least one storage location) and is durable. The data is eventually consistent across all storage locations, usually within one second or less. You can then choose to read this data in an eventually or strongly consistency fashion.

Concurrent writes to the same item should be handled with optimistic concurrency, you can do conditional writes using the DynamoDB Transaction Library (available in the AWS SDK for Java).

If you need to update more than one item atomically, you can use DynamoDB transactions.

DynamoDB transactions provide developers atomicity, consistency, isolation, and durability (ACID) across one or more tables within a single AWS account and region. You can use transactions when building applications that require coordinated inserts, deletes, or updates to multiple items as part of a single logical business operation.

https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/

CRUD operations are atomic; however, the official documentation says nothing about them being isolated (outside of DynamoDB transactions). In principle, race conditions can potentially occur and conditional updates can return an error.

See more here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html

Alternatively, your use case may benefit from DynamoDB global tables which uses “last writer wins” reconciliation between concurrent writes.

Upvotes: 6

Nadav Har'El
Nadav Har'El

Reputation: 13731

This question has been asked here many times in the past, see for example

Are dynamodb update expressions strongly consistent?

Are DynamoDB conditional writes strongly consistent?

The above accepted answer more-or-less suggests that in reads that happen as part of writes all bets are off, and you can't trust them to be consistent - so you need to use the new "DynamoDB transactions" feature. But I believe that the conclusion in that answer is wrong. The new "DynamoDB transactions" are needed when you have a transaction which needs to isolate writes to several different items, and to safely support failed non-idempotent writes. But in the single-item updates supported by UpdateItem, I believe the reads-before-write involved are in fact strongly consistent:

Unfortunately, DynamoDB's documentation isn't completely clear about this, but here is some evidence from DynamoDB's documentation to back up my belief:

  1. There are several reasons why an UpdateItem might need to read the old value of the item (a so-called read-before-write) - and the question is about the consistency of these read. One of the reasons to read is an UpdateExpression (e.g., an update can increment an attribute), another is a ConditionExpression (an update can be conditional on an old value of an attribute), and a third case is ReturnValues - the user asking to get the old value of the item. Even though the UpdateItem documentation is not clear about the consistency in the first two cases, it is very explicit about the consistency in the third case: "The values returned are strongly consistent.". I don't see any reason why the read-before-write in this case, but not others, would be strongly-consistent, so I believe it is strongly-consistent in all three cases.
  2. Several presentations of DynamoDB internals from DynamoDB explain how it works internally: It keeps three replicas for each piece of data, and that at any time one of the three is considered a "leader"; Writes always go to the leader first and succeed when the leader and a second replica have succeeded, eventual-consistency reads go to one of the three replicas at random (so may read from one replica that hasn't received a write yet) but strongly-consistent reads always go to the leader. This means that strongly-consistent reads and writes go to the same replica (the leader), and have the same consistency guarantees. Moreover, the leader linearizes the writes to this item (i.e., does them in some order, not concurrently), so the read-before-write of one write can see everything that was written in previous writes to the same item.

Upvotes: 6

Related Questions