CODEWITHSUNDEEP
javaandroidsearchamazon-dynamodb
Tom Reeve
Tom Reeve

Reputation: 107

AWS DynamoDB multiple "NE" string filters?

I'm trying to scan/query an AWS DynamoDB table for a list of items where id (a single string) is not equal to any of strings A, B, C, D, etc.

I've tried something like this:

for (String itemString : items) {
  scanExpression.addFilterCondition("id", 
      new Condition().withComparisonOperator(ComparisonOperator.NE)
      .withAttributeValueList(new AttributeValue().withS(itemString)));
}

PaginatedScanList<Items> result = mapper.scan(Item.class, scanExpression);

What appears to happen is that each time I add filter, it overwrites the previous values, so that the scan only checks against one itemString, not several. Am I missing something, or is this a limitation of DynamoDB?

Note: if it matters, in my example, I listed four values (A, B, C, D) but in production, this may be a list of hundreds of values.

Upvotes: 2

Views: 2969

Answers (1)

mkobit
mkobit

Reputation: 47249

You are correct in saying that the way you are doing it "overwrites the previous values". Here is the relevant code from the 1.9.23 SDK on DynamoDBScanExpression:

public void addFilterCondition(String attributeName, Condition condition) {
    if ( scanFilter == null )
        scanFilter = new HashMap<String, Condition>();

    scanFilter.put(attributeName, condition);
}

Since your attributeName will be id for both puts, the last value will win in this case.

In my opinion, this is poor behavior from DynamoDBScanExpression, and I would even lean more towards saying it is a bug that should be reported. The documentation does not state anything about when a duplicate attributeName is added and the method name makes it seem like this is unexpected behavior.

I don't see a good way to work around this other than building out the entire filter expression.

Another Note: On the documentation, I don't see a length constraint for how long a filter expression can be as well as how many ExpressionAttributeNames and ExpressionAttributeValues are allowed on a request. That may come into account if you are trying to filter out a ton of attribute values, but I haven't found any documentation of what that limit might be or what behavior you should expect.

StringJoiner filterExpression = new StringJoiner(" AND ");
Map<String, AttributeValue> expressionAttributeValues = new HashMap<>();
int itemAttributeSuffix = 0;
for (String itemString : items) {
    StringBuilder expression = new StringBuilder("#idname")
            .append(" ")
            .append("<>")
            .append(" ")
            .append(":idval")
            .append(itemAttributeSuffix);
    filterExpression.add(expression);
    expressionAttributeValues.put(":idval" + itemAttributeSuffix++,
        new AttributeValue().withS(itemString));
}
Map<String, String> expressionAttributeNames = Collections.singletonMap("#idname", "id");
scanExpression.setFilterExpression(filterExpression.toString());
scanExpression.setExpressionAttributeNames(expressionAttributeNames);
scanExpression.setExpressionAttributeValues(expressionAttributeValues);

PaginatedScanList<Items> result = mapper.scan(Item.class, scanExpression);

Upvotes: 1

Related Questions