CODEWITHSUNDEEP
amazon-web-servicesnosqlamazon-dynamodb
Azamat Abdullaev
Azamat Abdullaev

Reputation: 806

DynamoDB Scan eventual consistency vs. GSI Query eventual consistency

I have a table, which previously I have been using the Scan API (with "ConsistentRead": false, i.e eventually consistent) for my purpose of filtering by a non-key attribute. Because it is quite costly, now I have decided to add a global secondary index (GSI), in which the attribute I was scanning for before is a hash key, and I am going to use the Query API on this index.

I know that GSI's don't support "read after write" consistency, or strong consistency. I know that, when I was using the Scan API, I was not using this consistency as well, because I was doing it in an eventually consistent manner.

I am worried, does the time spent reflecting previous updates or writes in the GSI, differ much from the time spent reflecting updates and writes while scanning the table in an eventually consistent manner?

For example, is the following possible:

a) Scan

  1. A PutItem API was called on the table, i.e new record has been inserted.
  2. I scanned the table, after 50ms I got the fresh data.

b) Query a GSI

  1. A PutItem API was called on the table, i.e new record has been inserted.
  2. I queried the index, after 500ms I got the fresh data.

So, as you can see 500ms vs. 50ms difference is too large.

Can we have such a big difference in their consistency? Or, will we have quite similar values? I am concerned if this change won't break the existing functionality.

Upvotes: 4

Views: 716

Answers (1)

aherve
aherve

Reputation: 4070

Didn't test it myself, but 500ms seems a bit slow to me.

Did you see that the documentation warns about properly provisionning the table so that both the main table and index can be updated ?

Also, they mention "a fraction of a second" as an expected delay.

That being said, you can't be wrong to transition to a GSI. DynamoDB scans are really bad practice and will only get worse as your table grows. GSI performances do not depend on your table size

Upvotes: 1

Related Questions