CODEWITHSUNDEEP
multithreadingamazon-dynamodbboto3amazon-dynamodb-streams
humble_hustler
humble_hustler

Reputation: 23

How to perform concurrent writes on a DynamoDB table?

I want to stream data from different sources- Twitter and reddit, specifically, and store it onto a DynamoDB table. I am quite new to this, so I am unsure if it is doable. I was thinking of using 2 different threads, one for Twitter and one for Reddit. Both the threads will fetch data from the corresponding stream and insert into the DynamoDB table.

Does this seem feasible? Would it be possible to do concurrent writes in DynamoDB? If not, what could be an alternative approach to simultaneously store data from different sources on a single DynamoDB table?

Upvotes: 1

Views: 3496

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269490

It is highly-encouraged to perform concurrent writes to DynamoDB.

When an Amazon DynamoDB table is created, you can specify the Read and Write throughput per second. To fully utilize this capacity, you could use multiple threads on multiple servers.

To obtain the best throughput from DynamoDB, ensure that the writes employ wide-spread Partition Keys, since these identify how the data is partitioned across multiple DynamoDB servers. Each partition has a subset of the capacity. If many of the reads/writes use the same Partition Key, throughput will be reduced because the requests are hitting a hot partition.

Upvotes: 1

Related Questions