CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-dynamodb
hounded
hounded

Reputation: 726

AWS dynamodb loop putItem vs batchWrite

Hi I was just wondering if someone could give me clarification on the benefits of batchwrite

If we have let's say 36 items we want to write to dynamodb, i'm using a AWS lambda function and the way I see it I have two options (Pseudo code)

Option One

for item in items: 
    putItem(item)

Option two

for item in items:
    if cnt < 25
       batch.push(item)
    if cnt == 25 
       batchWrite(batch)
       cnt = 0
    cnt++

I feel like option one is quick and dirty but if my items would rarely go over 100 is it that bad (would I time out my lambda etc ..) ?

Anyway best practice clarification on this would be great.

Upvotes: 9

Views: 6508

Answers (1)

v.ng
v.ng

Reputation: 794

For both of your options, you should implement some kind of parallel send for better controlling and performance, like Promise.allSettled in JS and asyncio parallel in python.

Like mentioned in the comments aboce, using batchWrite does in fact reduce the number of requests called, but one critical limitation is it only accepts up to 25 requests and 16 MB total size in one batch requests , if you expect the requests will exceed this limitation, you need to, and you should implement a way to divide the requests into multiple batches, so that each batch size is under 25 requests and 16 MB.

Using putItem is more simple than using batchWrite, as it doesn't have the limitation mentioned above. But again, you will initiate a lot of API requests.

Both the method does not affect the cost, as AWS does not charge you by the number of API called to them, according to their pricing description, but the actual data write to and read from the DynamoDB table, which are known are WCU and RCU respectively. Btw, data transfer in is not charged.

In conclusion, for both putItem and batchWrite, what you have to concern is (1) how to implement the requests and handle retry incase of error. (2) The quantity of the record to be inserted.

Upvotes: 4

Related Questions