CODEWITHSUNDEEP
amazon-dynamodbboto3
Sean Pannella
Sean Pannella

Reputation: 383

Query DynamoDB with a hash key and a range key with Boto3

I am having trouble using AWS Boto3 to query DynamoDB with a hash key and a range key at the same time using the recommend KeyConditionExpression. I have attached an example query:

import boto3
from boto3 import dynamodb
from boto3.session import Session
dynamodb_session = Session(aws_access_key_id=AWS_KEY,
              aws_secret_access_key=AWS_PASS,
              region_name=DYNAMODB_REGION)
dynamodb = dynamodb_session.resource('dynamodb')
table=dynamodb.Table(TABLE_NAME)
request = {
    'ExpressionAttributeNames': {
        '#n0': 'hash_key',
        '#n1': 'range_key'
    },
    'ExpressionAttributeValues': {
        ':v0': {'S': MY_HASH_KEY},
        ':v1': {'N': GT_RANGE_KEY}
    },
    'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
    'TableName': TABLE_NAME
}
response = table.query(**request)

When I run this against a table with the following scheme:

Table Name: TABLE_NAME
Primary Hash Key: hash_key (String)
Primary Range Key: range_key (Number)

I get the following error and I cannot understand why:

ClientError: An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: Incorrect operand type for operator or function; operator or function: >, operand type: M

From my understanding the type M would be a map or dictionary type and I am using a type N which is a number type and matches my table scheme for the range key. If someone could explain why this error is happening or I am also open to a different way of accomplishing the same query even if you cannot explain why this error exists.

Upvotes: 38

Views: 67669

Answers (3)

James Rocker
James Rocker

Reputation: 1908

For anyone looking at using boto3 to query Dynamodb with composite keys in 2023 this is the first result from google.

To return a single record, the easiest way is to just pass the hash_key and range_key as a dictionary to the key argument on the get_item method.

e.g.

from boto3 import resource

dynamodb_table = resource("dynamodb", region_name="eu-west-1").Table("your_table_name")
dynamodb_table.get_item(Key={"hash_key_name": "value", "range_key_name": "value"})

For returning more than one object, I recommend using the query method and using the KeyConditionExpression argument. Then using the arguments from the boto3.dynamodb.conditions class to build your query. The example below giving the same results as the example above.

For example

from boto3 import resource
from boto3.dynamodb.conditions import Key

dynamodb_table = resource("dynamodb", region_name="eu-west-1").Table("your_table_name")
dynamodb_table.query(
    KeyConditionExpression=Key("hash_key_name").eq("value")
    & Key("range_key_name").eq("value")
)

The key difference is get_item, results are stored in "Item" key, while query results are in Items and are stored as a list

Upvotes: 2

Imtiaz
Imtiaz

Reputation: 101

Adding this solution as the accepted answer did not address why the query used did not work.

TLDR: Using query on a Table resource in boto3 has subtle differences as opposed to using client.query(...) and requires a different syntax.

The syntax is valid for a query on a client, but not on a Table. The ExpressionAttributeValues on a table do not require you to specify the data type. Also if you are executing a query on a Table resource you do not have to specify the TableName again.

Working solution:

from boto3.session import Session

dynamodb_session = Session(aws_access_key_id=AWS_KEY,aws_secret_access_key=AWS_PASS,region_name=DYNAMODB_REGION)

dynamodb = dynamodb_session.resource('dynamodb')
table = dynamodb.Table(TABLE_NAME)

request = {
    'ExpressionAttributeNames': {
        '#n0': 'hash_key',
        '#n1': 'range_key'
    },
    'ExpressionAttributeValues': {
        ':v0': MY_HASH_KEY,
        ':v1': GT_RANGE_KEY
    },
    'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
}
response = table.query(**request)

I am the author of a package called botoful which might be useful to avoid dealing with these complexities. The code using botoful will be as follows:

import boto3
from botoful import Query

client = boto3.Session(
    aws_access_key_id=AWS_KEY,
    aws_secret_access_key=AWS_PASS,
    region_name=DYNAMODB_REGION
).client('dynamodb')

results = (
    Query(TABLE_NAME)
        .key(hash_key=MY_HASH_KEY, range_key__gt=GT_RANGE_KEY)
        .execute(client)
)

print(results.items)

Upvotes: 6

Filipe Felisbino
Filipe Felisbino

Reputation: 2912

The Boto 3 SDK constructs a Condition Expression for you when you use the Key and Attr functions imported from boto3.dynamodb.conditions:

response = table.query(
    KeyConditionExpression=Key('hash_key').eq(hash_value) & Key('range_key').eq(range_key_value)

)

Reference: Step 4: Query and Scan the Data

Hope it helps

Upvotes: 73

Related Questions