CODEWITHSUNDEEP
javascriptamazon-web-servicesaws-lambdaamazon-dynamodbserverless
CeraMix
CeraMix

Reputation: 121

DynamoDB batchWrite not awaiting before moving on, and no errors?

I have a lambda that is expected to iterate through the scanned items, batch each item within 25 item batches, create a DeleteRequest from these items and delete them using Dynamo's .batchWrite() method.

I'm having trouble with DynamoDB's dynamoDb.batchWrite({ RequestItems: requestItems }).promise() method, where it seems to never finish before moving to the next iteration of my batch of 25 loop. I think this is an issue with my async/await, however, I cannot seem to spot it!

The output of this lambda running is like below:

enter image description here

I have a method that scans the items:

async function getAllItemsFromTable (TableName) {
  const res = await dynamoDb.scan({ TableName }).promise()
  return res.Items
}

A batch function (so each request is on 25 items, to satisfy requirements):

function batch(array, size) {
  const chunked_arr = []
  let index = 0
  while (index < array.length) {
    chunked_arr.push(array.slice(index, size + index))
    index += size
  }
  return chunked_arr
}

A delete function (that processes these batches by writing to Dynamo):

async function deleteAllItemsFromTable (items) {
  let numItemsDeleted = 0
  const batchedArr = batch(items, 25)

  batchedArr.forEach(async (chunk) => {
    const requestItems = {
      [tableName]: chunk.map((item) => {
        numItemsDeleted++
        const Key = { id: item.id }
        return { DeleteRequest: { Key } }
      }),
    }
    
    if (requestItems[tableName].length > 0) {
      console.log("requestItems", requestItems)
      try {
        console.log("Starting Batch Write..") // this prints.
        // this line below NEVER finishes. It also doesn't spit any errors out. I'm awaiting, what else?
        await dynamoDb.batchWrite({ RequestItems: requestItems }).promise()
        console.log("Finished Batch Write..") // this doesn't ever print.
      } catch (error) {
        console.log("Error: ", error) // this doesn't ever print.
      }
    }
  })

  console.log("numItemsDeleted", numItemsDeleted) // this prints.
  return { numItemsDeleted }
}

And finally, I run my lambda like so:

const items = await getAllItemsFromTable(tableName)
const { numItemsDeleted } = await deleteAllItemsFromTable(items)
console.log(`--- ${numItemsDeleted} items deleted`)

Upvotes: 1

Views: 588

Answers (1)

Warren Parad
Warren Parad

Reputation: 4091

The problem is you are never awaiting the batch list loop execution. You start all the executions and then move to the next line. So you end up never awaiting the executions.

Change:

batchedArr.forEach(async (chunk) => {...})

to

await Promise.all(batchedArr.map(async (chunk) => {...})))

Upvotes: 1

Related Questions