CODEWITHSUNDEEP
amazon-web-servicesamazon-kinesisaws-sdk-nodejsamazon-kcl
superB
superB

Reputation: 288

Maintaining order using Amazon's Kinesis with atleast 1000 messages per sec

I am trying to send 1000's of messages in sequence using Amazon's Kinesis but messages being received by kinesis consumer is not in sequence. Things I have tried :

  1. Used putRecord() method to send with SequenceNumberForOrdering. Achieved ordered sequence with this but throughput is very less.
  2. Used putRecords method to send but not successful.
  3. Used KCL + Amazon's API consumer.

Upvotes: 1

Views: 621

Answers (1)

ketan vijayvargiya
ketan vijayvargiya

Reputation: 5659

Kinesis maintains sequence within shards, not across all shards in a stream. I am guessing you have more than 1 shard in your stream and hence observing the read records out of sequence. (You can also quickly validate this by repeating #2 with just 1 shard in your stream.)

Quote from here:

please note that Record Sequence Numbers (RSNs) are not globally tracked in the stream and are instead only tracked per shard.

Upvotes: 1

Related Questions