CODEWITHSUNDEEP
javaamazon-web-servicestime-seriesdata-visualizationamazon-dynamodb
Marsellus Wallace
Marsellus Wallace

Reputation: 18601

Using DynamoDB for Timeseries Data with visualization goal

I've been advised to look into DynamoDB to store timeseries data but I'm not quite sure about it given that my final goal is data visualization.

I have sensors that send data once every 10 minutes and I'd like to visualize the data in some charts with a weekly view by default (1008 data points (datetime/values) per week). Let's suppose that I provision 10,000 Reads/Second (AWS 'default' maximum) and let's assume that 1 record will fit in 1 unit of capacity (1kb).

Besides stuff getting expensive, does this mean that I cannot even support only 10 clients simultaneously? Am I wrong or DynamoDB is just not the right tool for the job?

Upvotes: 2

Views: 1861

Answers (1)

Guy
Guy

Reputation: 12901

DynamoDB is very good to store your in coming event data, but it should not be the only tool for you to work with. You can integrate DynamoDB with other tools:

  • Put a cache (ElasticCache, for example) in front of your DynamoDB to allow serving your repeated queries from it, instead of DynamoDB
  • Put a buffer queue (SQS, for example) in front of your DynamoDB to allow your sensors to send their reports in various rates, while keeping a lower balanced rate of writes into your DynamoDB.

You can also have multiple formats of your data inside DynamoDB., based on your access pattern. For example, you can have a single record that holds the data points of the whole week per a sensor, and you can update this single record every 10 minutes, instead of only appending new record data every report. This weekly record per sensor, can be daily or monthly as you see fit. Still you will only have to read 1 or 7 or any other small number of records per write and read.

Updated with link to more on DynamoDB table design from DynamoDB documentations: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GuidelinesForTables.html

Upvotes: 9

Related Questions