CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-dynamodb
Tai
Tai

Reputation: 1254

DynamoDb stored object duplicates

I wrote this function to see if an object has already been stored in dynamo.

def doesIDExist(key):
    print(key)
    info=key.split("/")
    #set properties
    try:
        id=info[1]+info[2]
        results = dynamoConn.query(swfTable,id)
        if results is not None:
            return True
    except:
        return False
    return False

I call a check before writing to any db:

 if(dynamoStorage.doesIDExist(theNewKey)==False):

      data= recursiveInnerFile(fPath,theNewKey)
      images = data[0]
      text = data[1]
      dynamoStorage.createSwfInDynamo(theNewKey,images,text)

and yet I still have duplicates showing up. What am I missing?

EDIT:

I'm not getting errors: I am getting a table object and my function is returning True. But nothing is being added?

Second Edit: I solved this below.

Upvotes: 0

Views: 2049

Answers (2)

Tai
Tai

Reputation: 1254

I solved this. The problem was that the object returned was not a table. I needed to get the table with .table and then get the count .item_count. 

def doesIDExist(key):
    print(key)
    info=key.split("/")
    #set properties
    #try:
    id=info[1]+info[2]
    results = dynamoConn.query(swfTable,id).table.item_count
    print results
    if results > 0:
        return True
    #except:
    #    return False
    return False

Upvotes: 0

Erben Mo
Erben Mo

Reputation: 3634

To avoid duplication the best way is to use "Conditional Put" so that if an item with the key already exists, the new item will not overwrite that existing item.

see the doc for python

Create a new item or replace an old item with a new item (including all attributes). If an item already exists in the specified table with the same primary key, the new item will completely replace the old item. You can perform a conditional put by specifying an expected rule.

some examples

you just need need to pass the 'expected' as a dictionary to your put_item call. 

expected={
    'yourHashKeyName': {'Exists': False}
}

The reason your code above doesn't work is dynamoConn.query will always return a TableGenerator object even when there is no item with that hash key.

The query function takes a Count parameter which returns "a total number of items for the Query operation, even if the operation has no matching items for the assigned filter."

results = dynamoConn.query(swfTable,id, count = True)

and then check

results.count > 0

if that id already exists, results.count should be 1. otherwise it should be 0.

Another option is to use get_item API. your table doesn't have a range key and you can call get_item with the "id" as hashkey.

Upvotes: 2

Related Questions