CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbreporting
mute
mute

Reputation: 311

Temporarily storing report data onto AWS DynamoDB? Is this a viable solution?

I have a web application in which the users can run reports on data. The problem is that the queries are so slow because of the data we are working with.

I was thinking of making the process asynchronous and have another machine work on that thread: the querying of that data, putting it into AWS DynamoDB, and then notifying the users.

The data is flat for the most part (just rows and columns). Usually, the record size will range around in the 100s, sometimes 1000s, and rarely in the 10,000s.

Is this is a good idea? What would the performance be like?

Upvotes: 2

Views: 1201

Answers (2)

John Veldboom
John Veldboom

Reputation: 2267

Similar to what @Scrappydog mentioned. You could do a mix of DynamoDB and S3.

  1. DynamoDB would store the query meta data - like user id and S3 file names
  2. S3 would store the actual query results in files

This would allow you to use DynamoDB to tie all the moving pieces together. Also would keep large writes out of DynamoDB to keep the cost down.

Upvotes: 1

Scrappydog
Scrappydog

Reputation: 2874

Classic architecture answer: "It depends"

  1. If your long running query is on other Dynamo table(s) then storing the results in Dynamo makes some sense. (familiar technology that you are already using).

  2. If your query is running against some other database then why not store your results in a new table in that database, or a new dedicated database on the same platform?

  3. Or keep it simple and just write your results to a file in S3 (json, xml, csv, whatever...)?

Upvotes: 1

Related Questions