CODEWITHSUNDEEP
amazon-dynamodb
Sten
Sten

Reputation: 1109

DynamoDb scan with ExclusiveStartKey returns zero items on first run then more than zero items

I am doing a scan on a DynamoDb table. The strange thing is that it returns zero items on the first call, and then more than zero items when I provide the last evaluated key to the ExclusiveStartKey parameter.

let lastEvaluatedKey;

do {
  const { Items, LastEvaluatedKey } = await documentClient.scan({
    TableName: "myTable",
    FilterExpression: "begins_with(pk, :prefix)",
    ExpressionAttributeValues: {
      ":prefix": "something#",
    },
    ExpressionAttributeNames: {
      "#type": "type",
    },
    ProjectionExpression: "pk",
    Limit: 10, 
    ExclusiveStartKey: lastEvaluatedKey,
  });

  console.log(`Scanned and found ${Items?.length ?? 0} items`);

  lastEvaluatedKey = LastEvaluatedKey;
} while (lastEvaluatedKey);

Output from above is

Scanned and found 2 items.

I am 100% certain that no rows were inserted between the calls. The pattern is consistent but with different numbers of calls with 0 items. I would have expected the 2 items to be returned on the first call. What is going on?

Upvotes: 0

Views: 475

Answers (1)

hunterhacker
hunterhacker

Reputation: 7132

You have some number of partitions. Some partitions are empty or don’t have items that match your filter. Your scan happens to start with an empty/unmatching partition. Single calls don’t cross partition boundaries, so your first retrieval reads from the empty/unmatching partition then your next call reads from the next partition.

Upvotes: 2

Related Questions