CODEWITHSUNDEEP
amazon-web-servicesnosqlamazon-dynamodbaws-lambda
T1000
T1000

Reputation: 423

How to update all records in DynamoDB?

I am new to nosql / DynamoDB.

I have a list of ~10 000 container-items records, which is updated every 6 hours:

[
   { containerId: '1a3z5', items: ['B2a3, Z324, D339, M413'] },
   { containerId: '42as1', items: ['YY23, K132'] },
   ...
]

(primary key = containerId)

  1. Is it viable to just delete the table, and recreate with new values?

  2. Or should I loop through every item of the new list, and conditionally update/write/delete the current DynamoDB records (using batchwrite)?

Upvotes: 3

Views: 10594

Answers (2)

Samantha Atkins
Samantha Atkins

Reputation: 678

If the update touches some but not all existing items and if partial update of 'items' is possible then you have no choice but to do a per record operation. And this would be true even with a more capable database.

You can perhaps speed it up by retrieving only the existing containerIds first so based on that set you know which to do update versus insert on. Alternately you can do a batch retrieve by ids using the ids from the set of updates and which every ones do not return a result are the ones you have to insert and the ones where you do are the ones to update.

Upvotes: 0

Bojan Trajkovski
Bojan Trajkovski

Reputation: 1176

For this scenario batch update is better approach. You have 2 cases:

  • If you need to update only certain records than batch update is more efficient. You can scan the whole table and iterate thought the records and only update certain records.
  • If you need to update all the records every 6 hours batch update will be more efficient, because if you drop the table and recreate table, that also means you have to recreate indexes and this is not a very fast process. And after you recreate table you still have to do the inserts and in the meantime you have to keep all the records in another database or in-memory.

One scenario where deleting the whole table is a good approach if you need to delete all the data from the table with thousands or more records, than its much faster to recreate table, than delete all the records though API.

And one more suggestion have you considered alternatives, because your problem does not look like a good use-case for DynamoDB. For example MongoDB and Cassandra support update by query out of the box.

Upvotes: 1

Related Questions