Reputation: 441
I am a bit new to dynamodb
See error I get when trying to get the max id of my dynamodb table in python lambda function using instructions in below StackOverflow post in below link Dynamodb max value
An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: The expression can not be empty;\"}"
see my lambda function code below
import json
import boto3
TABLE_NAME = 'user-profiles'
dynamo_DB = boto3.resource('dynamodb')
def lambda_handler(event, context):
user_id = event['user_id']
email = event['email']
bvn = event['bvn']
password = event['password']
phone = event['phone']
gender = event['gender']
output = ''
if len(user_id) > 1 and len(password) > 5:
try:
table = dynamo_DB.Table(TABLE_NAME)
values = list(table.query(
KeyConditionExpression='',
ScanIndexForward=False,
Limit=1
)
)
max_id = values[0]['id']
new_id = max_id + 1
Item = {
'id': str(new_id),
'profile-id': str(new_id),
'user_id': user_id,
'email': email,
'bvn': bvn,
'password': password,
'phone': phone,
'gender': gender
}
table.put_item(Item=Item)
output += 'Data Inserted To Dynamodb Successfully'
except Exception as e:
output += 'error with dynamo registration ' + str(e)
# print(output)
else:
output += 'invalid user or password entered, this is ' \
'what i received:\nusername: ' \
+ str(user_id) + '\npassword: ' + str(password)
return {
"statusCode": 200,
"body": json.dumps({
"message": output,
}),
}
# print(output)
Upvotes: 0
Views: 3892
Reputation: 367
You cannot query
with empty KeyConditionExpression
, if you need to read all records from the table you need to use scan
. But you cannot use ScanIndexForward
there to order records forward.
Seems like you're trying to implement primary key incrementation. I want to warn you, your solution is not really awesome, because you easily can hit a race condition.
What I would suggest:
I guess you are using id
as a primary key (aka partition key). it's okay. what I would do is upsert an extra record in the table, with say increment
value:
increment = table.update_item(
Key={'id': 'increment'},
UpdateExpression='ADD #increment :increment',
ExpressionAttributeNames={'#increment': 'increment'},
ExpressionAttributeValues={':increment': 1},
ReturnValues='UPDATED_NEW',
)
new_id = increment['Attributes']['increment']
This query will update the existing record with id: 'increment'
and store a new incremented number in the record, if it is the very first query the record will be created with increment: 1
and subsequent calls will increment it. ReturnValues means the query will return the result after the update and you will get a new id.
put the code in place instead of where you query
the last record
so your code would look like:
import json
import boto3
TABLE_NAME = 'user-profiles'
dynamo_DB = boto3.resource('dynamodb')
def lambda_handler(event, context):
user_id = event['user_id']
email = event['email']
bvn = event['bvn']
password = event['password']
phone = event['phone']
gender = event['gender']
output = ''
if len(user_id) > 1 and len(password) > 5:
try:
table = dynamo_DB.Table(TABLE_NAME)
increment = table.update_item(
Key={'id': 'increment'},
UpdateExpression='ADD #increment :increment',
ExpressionAttributeNames={'#increment': 'increment'},
ExpressionAttributeValues={':increment': 1},
ReturnValues='UPDATED_NEW',
)
new_id = increment['Attributes']['increment']
Item = {
'id': str(new_id),
'profile-id': str(new_id),
'user_id': user_id,
'email': email,
'bvn': bvn,
'password': password,
'phone': phone,
'gender': gender
}
table.put_item(Item=Item)
output += 'Data Inserted To Dynamodb Successfully'
except Exception as e:
output += 'error with dynamo registration ' + str(e)
# print(output)
else:
output += 'invalid user or password entered, this is ' \
'what i received:\nusername: ' \
+ str(user_id) + '\npassword: ' + str(password)
return {
"statusCode": 200,
"body": json.dumps({
"message": output,
}),
}
# print(output)
and you're good.
Extra thoughts:
And to be 100% sure that there is no race condition on incrementation, you can implement a locking mechanism this way: Before incrementing, put an extra record with id
value lock
and lock
attribute with any value, and use ConditionExpression='attribute_not_exists(lock)'
. Then make an increment and then release the lock by removing the record lock
. So while the record is there the second attempt to 'make a lock' would break by the condition that attribute lock
exists and throw error ConditionalCheckFailedException
(you can catch the error and show to a user that the record is locked or whatever.)
Here is an example in JavaScript sorry:
module.exports.DynamoDbClient = class DynamoDbClient {
constructor(tableName) {
this.dynamoDb = new DynamoDB.DocumentClient();
this.tableName = tableName;
}
async increment() {
await this.lock();
const {Attributes: {increment}} = await this.dynamoDb.update({
TableName: this.tableName,
Key: {id: 'increment'},
UpdateExpression: 'ADD #increment :increment',
ExpressionAttributeNames: {'#increment': 'increment'},
ExpressionAttributeValues: {':increment': 1},
ReturnValues: 'UPDATED_NEW',
}).promise();
await this.unlock();
return increment;
}
async lock(key) {
try {
await this.dynamoDb.put({
TableName: this.tableName,
Item: {id: 'lock', _lock: true},
ConditionExpression: 'attribute_not_exists(#lock)',
ExpressionAttributeNames: {'#lock': '_lock'},
}).promise();
} catch (error) {
if (error.code === 'ConditionalCheckFailedException') {
throw new LockError(`Key is locked.`);
}
throw error;
}
}
unlock() {
return this.delete({id: 'lock'});
}
async delete(key) {
await this.dynamoDb.delete({
TableName: this.tableName,
Key: key,
}).promise();
}
}
// usage
const client = new DynamoDbClient('table');
const newId = await client.increment();
...
Upvotes: 1