CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodb
Dengke Liu
Dengke Liu

Reputation: 49

AWS Data store choice

I am working on a project to create an application. The application will check a input against a list of values, and return response. You can think like I am verifying if a student has membership with a gym.

In my use case, the size list of values could be small as thousands and huge as millions. Also, I may want to daily update the list, or even remove the list and create a new one from my data source.

What would be the best AWS data store I can use in this cause? I am thinking about using dynamo DB.

Upvotes: 0

Views: 60

Answers (2)

peter bray
peter bray

Reputation: 1345

DynamoDB may be good for your use case. However, you mention something which makes me question that:

In my use case, the size list of values could be small as thousands and huge as millions. Also, I may want to daily update the list, or even remove the list and create a new one from my data source.

Unlike in a traditional SQL-like environment with RDS/MySQL/Aurora, in DynamoDB you cannot do a single query to delete or update all or some of the items.

Indeed, if you have millions of items that need updating or deleting, then DynamoDB may not be the best choice for you. This is because in DynamoDB each item must be individually updated or deleted.

Moreover, DynamoDB charges you per "write worker," and each write worker can perform one delete/update per second. As a result, if you have millions of rows that need deleting or updating, you will need to have sufficient workers to do this. This can be expensive, particularly if these changes occur "all at once" (versus spread throughout the day).

Obviously you can delete and update items in Dynamo. You can leverage DynamoDB Streams to process new items; you can add a time-to-live so that records delete after 48 hours (minimum); you can use auto-scaling to grow your workers to handle predictable increases. Or you can use SQS or other techniques to queue and piecemeal your updates over time.

However, these all require careful design (much more so than traditional RDS in my opinion). If you have an update/delete heavy application, I recommend using a relational database.

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 269350

If your use case is "Does this entry exist within this table?", then DynamoDB is a great choice because it would return a result very quickly and there is no infrastructure to manage.

You could also go totally serverless using AWS API Gateway and AWS Lambda, so your whole application can operate without servers and can automatically scale based upon load.

Other choices are:

  • A traditional SQL database using Amazon RDS, but this would be overkill for the stated use-case
  • Objects stored in Amazon S3, and simply query whether the object exists. (You can think of Amazon S3 as a very large NoSQL database.)

Upvotes: 1

Related Questions