CODEWITHSUNDEEP
pythonamazon-dynamodbboto3
na1368
na1368

Reputation: 127

Error doing update_item in DynamoDB python boto3

I have a table in DynamoDB that holds a bunch of documents for me. The documents have the following fields: computedID (Primary partition key), publication_timestamp, displayTitle, displayText, displayUrl, producer and tags.

I'd like to do update_item on the table, so that a record gets updated only if any of the fields publication_timestamp, displayTitle, displayText, displayUrl, producer and tags has changed. If the record is totally new it will be simply inserted in the table.

The problem is that not all the existing documents in the table or the incoming documents have displayTitle, displayText, displayUrl and tags. They may miss any number of these fields.

I have tried the following:

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('project_Incoming_Data')
print("Sending documents to DynamoDB...")

for item in Docs:
    try:
        response=table.update_item(
            Key={"computedID":item["computedID"]},
            UpdateExpression="SET publication_timestamp = :time, displayTitle= :title, displayText= :text, producer = :p, tags= :tags, displayUrl= :url, time_to_live= :ttl",
            ConditionExpression= "publication_timestamp <> :time OR (attribute_exists(displayTitle) AND displayTitle <> :title) OR (attribute_exists(displayText) AND displayText <> :text) OR producer <> :p OR (attribute_exists(tags) AND tags <> :tags) OR (attribute_exists(displayUrl) AND displayUrl <> :url)",
            ExpressionAttributeValues={
                    ":time":item["publication_timestamp"],
                    ":ttl":item["time_to_live"],
                    ":title":item["displayTitle"],
                    ":text":item["displayText"],
                    ":p":item["producer"],
                    ":tags":item["tags"],
                    ":url":item["displayUrl"]
            },
            ReturnValues="UPDATED_NEW"
            )
        print("response is: "+str(response))
    except Exception as e:
        print (e)
print("Done with sending documents to DynamoDB")

I still am unable to get some of my documents into DynamoDB. The error I get is 'displayText'! I am guessing that the mechanism I have in place for making sure that the field exists on the record doesn't work for documents that don't have that field.

Any idea how to fix this?

Upvotes: 2

Views: 2522

Answers (1)

na1368
na1368

Reputation: 127

I found the solution! The problem was that even though a document could miss any of the displayText, displayTitle, tags or displayUrl fields, still all three of UpdateExpression, ConditionExpression and ExpressionAttributeValues were considering those fields for the document. The solution is to construct them for each document separately based on the fields present in the document.

def send_docs_to_DynamoDB(Docs):
    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('Compete_Dental_Incoming_Data')
    print("Sending documents to DynamoDB...")

    for item in Docs:
            expression_attribute_values={
                    ":time":item["publication_timestamp"],
                    ":ttl":item["time_to_live"],
                    ":p":item["producer"]
            }
            update_expression="SET publication_timestamp = :time, producer = :p, time_to_live= if_not_exists(time_to_live, :ttl)"
            condition_expression= "publication_timestamp <> :time OR producer <> :p"
            try:
                    if 'displayTitle' in item.keys():
                            update_expression+=", displayTitle= :title"
                            expression_attribute_values[":title"]=item["displayTitle"]
                            condition_expression+=" OR displayTitle <> :title"
                    if 'displayText' in item.keys():
                            update_expression+=", displayText= :text"
                            expression_attribute_values[":text"]=item["displayText"]
                            condition_expression+=" OR displayText <> :text"
                    if 'displayUrl' in item.keys():
                            update_expression+=", displayUrl= :url"
                            expression_attribute_values[":url"]=item["displayUrl"]
                            condition_expression+=" OR displayUrl <> :url"
                    if 'tags' in item.keys():
                            update_expression+=", tags= :tags"
                            expression_attribute_values[":tags"]=item["tags"]
                            condition_expression+=" OR tags <> :tags"

                    response=table.update_item(
                            Key={"computedID":item["computedID"]},
                            UpdateExpression=update_expression,
                            ConditionExpression= condition_expression,
                            ExpressionAttributeValues=expression_attribute_values,
                            ReturnValues="UPDATED_NEW"
                            )
                    print("response is: "+str(response))
            except Exception as e:
                    print (e)
    print("Done with sending documents to DynamoDB")

Upvotes: 4

Related Questions