CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-dynamodb
Varun.Excelsior
Varun.Excelsior

Reputation: 21

Using s3 as a database rather than dynamodb

I have a whatsapp chatbot project in AWS. I'm using DynamoDB as the database for the system. I'm looking to port my data over to S3 thinking it'd be more affordable to store data in S3 rather than DynamoDB. But I'm not sure it'd be a good fit as I'd lose key constraints and other functionality of a DB.

I'm not even sure if it is a good idea as I'm thinking of scenarios where I'd have to read, update and delete rows. The idea of moving over to S3 was brought forward due to cost concerns.

Please tell me if moving over data of a project to S3 is a good idea. If so, what steps would be changed?

Thank you.

Edit: I'm thinking of writing the data from DynamoDB as json file in S3 rather than csv. I think I have to go through many steps to do a simple update in case of csv.

2nd edit: The idea turned out to be not the best way to go with. I have no querying capabilities, no way to update the file (have to re-write the whole file with all the data) and data retrieval latency. I'm sticking with dynamodb for now. I've been doing scans on the tables for a while now without exploring other querying methods. The charges spiked in Nov-Dec last year; I took a backup of the tables and deleted them in order to work on some other projects. Started working on it again since 2 weeks and haven't incurred any charges.

Upvotes: 2

Views: 3642

Answers (1)

best wishes
best wishes

Reputation: 6634

It is tempting to use s3 as data store, but you will hit limitations like below

  1. S3 response times are not consistent,as less as dynamodb. (Please do a performance test putting same data in s3 and dynamodb and do a fetch).

  2. There will not be any get all messages sort by last modified date support.

  3. There will not be a partial retrieval of json. unless you denormalize everything. Which will be a huge performance hit considering s3 response times are more than dynamodb.

  4. It is almost impossible to store a relational model into flat key value without any relationship among stored files apart from probably a file.

Considering the above points, I would not have used s3 as a data store for a relational use case.

Upvotes: 1

Related Questions