CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbaws-sdkdynamodb-queries
Cacho Santa
Cacho Santa

Reputation: 6914

DynamoDB QuerySpec {MaxResultSize + filter expression}

From the DynamoDB documentation

The Query operation allows you to limit the number of items that it returns in the result. To do this, set the Limit parameter to the maximum number of items that you want.

For example, suppose you Query a table, with a Limit value of 6, and without a filter expression. The Query result will contain the first six items from the table that match the key condition expression from the request.

Now suppose you add a filter expression to the Query. In this case, DynamoDB will apply the filter expression to the six items that were returned, discarding those that do not match. The final Query result will contain 6 items or fewer, depending on the number of items that were filtered.

Looks like the following query should return (at least sometimes) 0 records.

In summary, I have a UserLogins table. A simplified version is:

1. UserId - HashKey
2. DeviceId - RangeKey
3. ActiveLogin - Boolean
4. TimeToLive - ...

Now, let's say UserId = X has 10,000 inactive logins in different DeviceIds and 1 active login.

However, when I run this query against my DynamoDB table:

QuerySpec{
hashKey: null,
rangeKeyCondition: null,
queryFilters: null, 
nameMap: {"#0" -> "UserId"}, {"#1" -> "ActiveLogin"}
valueMap: {":0" -> "X"}, {":1" -> "true"}
exclusiveStartKey: null,
maxPageSize: null, 
maxResultSize: 10,
req: {TableName: UserLogins,ConsistentRead: true,ReturnConsumedCapacity: TOTAL,FilterExpression: #1 = :1,KeyConditionExpression: #0 = :0,ExpressionAttributeNames: {#0=UserId, #1=ActiveLogin},ExpressionAttributeValues: {:0={S: X,}, :1={BOOL: true}}}

I always get 1 row. The 1 active login for UserId=X. And it's not happening just for 1 user, it's happening for multiple users in a similar situation.

Are my results contradicting the DynamoDB documentation?

It looks like a contradiction because if maxResultSize=10, means that DynamoDB will only read the first 10 items (out of 10,001) and then it will apply the filter active=true only (which might return 0 results). It seems very unlikely that the record with active=true happened to be in the first 10 records that DynamoDB read.

This is happening to hundreds of customers that are running similar queries. It works great, when according to the documentation it shouldn't be working.

Upvotes: 1

Views: 803

Answers (1)

F_SO_K
F_SO_K

Reputation: 14829

I can't see any obvious problem with the Query. Are you sure about your premise that users have 10,000 items each?

Your keys are UserId and DeviceId. That seems to mean that if your user logs in with the same device it would overwrite the existing item. Or put another way, I think you are saying your users having 10,000 different devices each (unless the DeviceId rotates in some way).

In your shoes I would just remove the filterexpression and print the results to the log to see what you're getting in your 10 results. Then remove the limit too and see what results you get with that.

Upvotes: 0

Related Questions