CODEWITHSUNDEEP
mysqlamazon-dynamodb
Darryl Sawyer
Darryl Sawyer

Reputation: 13

Latest Partition Key entry in DynamoDB

so to explain my use case and where I am getting stuck:

I am replicating a Mysql DB to a DynamoDB instance. Once in DynamoDB the table looks like this:

dynamo table

The Partition key is record_index with a sort key of meterId.

What I am looking to do is tune my API that queries the Mysql DB and writes to the DynamoDB:

Query DynamoDB for last written record_index (highest number, as it is incremental in the Mysql DB), use that record_index result as the starting point to query the Mysql DB and then write the results to the DynamoDB table.

Effectively I am trying to create an API that only writes incremental data.

I have googled and googled and cannot find a scan or query that will result in what I need.

Any suggestions?

Upvotes: 0

Views: 640

Answers (1)

Renato Byrro
Renato Byrro

Reputation: 3784

When you add a new record in your table, include an attribute (e.g. "my_attr") that always has the same value (e.g. "constant").

Create a secondary index where my_attr is the primary key and record_index is the sort key. Then query the secondary index by my_attr='constant', limiting to one result in descending order. The item returned will have your last record_index.

Beware that this would create what is called a hot partition. If you need high throughput from this query or to insert/update data in the main table, this design might be a problem.

See more: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html

Upvotes: 1

Related Questions