CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbamazon-kinesis
Wonger
Wonger

Reputation: 447

DynamoDB CDC event ordering - Kinesis Data Streams vs DynamoDb streams

Reading the documentation for DynamoDB cdc streams, there is a table which lays out some of the differences between using Kinesis Data Streams and DynamoDB streams. The "Ordering of records" row says the following for Data Streams:

The timestamp attribute on each stream record can be used to identify the actual order in which changes occurred in the DynamoDB table.

vs DynamoDB Streams:

For each item that is modified in a DynamoDB table, the stream records appear in the same sequence as the actual modifications to the item.

I interpret this wording as DynamoDB streams guaranteeing ordering based on item modification order in dynamo where as this isn't guaranteed for Data streams. Is my interpretation correct?

Currently I'm using Data Streams for CDC, but need to make a new app which requires that records are in based on when they were inserted into dynamo. Can I continue to use Data Streams or should I enable DynamoDb streams as well?

Upvotes: 1

Views: 984

Answers (1)

Leeroy Hannigan
Leeroy Hannigan

Reputation: 19683

I interpret this wording as DynamoDB streams guaranteeing ordering based on item modification order in dynamo where as this isn't guaranteed for Data streams. Is my interpretation correct?

That is correct understanding. Item level ordering, which means modifications to a single item will be guaranteed to be in order of when the occurred.

Currently I'm using Data Streams for CDC, but need to make a new app which requires that records are in based on when they were inserted into dynamo. Can I continue to use Data Streams or should I enable DynamoDb streams as well?

I would use DynamoDB Streams as it streamlines having to maintain order across multiple shards of your stream.

Properties Kinesis Data Streams for DynamoDB DynamoDB Streams
Data retention Up to 1 year. 24 hours.
Kinesis Client Library (KCL) support Supports KCL versions 1.X and 2.X. Supports KCL version 1.X.
Number of consumers Up to 5 simultaneous consumers per shard, or up to 20 simultaneous consumers per shard with enhanced fan-out. Up to 2 simultaneous consumers per shard.
Throughput quotas Unlimited. Subject to throughput quotas by DynamoDB table and AWS Region.
Record delivery model Pull model over HTTP using GetRecords and with enhanced fan-out, Kinesis Data Streams pushes the records over HTTP/2 by using SubscribeToShard. Pull model over HTTP using GetRecords.
Ordering of records The timestamp attribute on each stream record can be used to identify the actual order in which changes occurred in the DynamoDB table. For each item that is modified in a DynamoDB table, the stream records appear in the same sequence as the actual modifications to the item.
Duplicate records Duplicate records might occasionally appear in the stream. No duplicate records appear in the stream.
Stream processing options Process stream records using AWS Lambda, Kinesis Data Analytics, Kinesis data firehose , or AWS Glue streaming ETL. Process stream records using AWS Lambda or DynamoDB Streams Kinesis adapter.
Durability level Availability zones to provide automatic failover without interruption. Availability zones to provide automatic failover without interruption.

Upvotes: 2

Related Questions