CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-dynamodbboto3nosql
Enrique Dev
Enrique Dev

Reputation: 119

Infinite loop in DynamoDB scan

As explained in Complete scan of dynamoDb with boto3 I build a solution to full scan a DynamoDB table using some condition. This is my code:

def dynamo_scan(table_name, params_dict):
    dynamo_table = boto3.resource('dynamodb').Table(table_name)
    response = do_scan(dynamo_table, params_dict)
    results = response['Items']
    while response.get('LastEvaluatedKey'):
        print "Iterating"
        print response.get('LastEvaluatedKey')
        params_dict['ExclusiveStartKey'] = response['LastEvaluatedKey']
        response = do_scan(dynamo_table, params_dict)
        results.extend(response['Items'])
    return results

def do_scan(dynamo_table, params_dict):
    return dynamo_table.scan(**params_dict)

But when the code is executed it gets into an infinite loop. This is the output for the each iteration of LastEvaluatedKey:

{u'my_id': u'1698', u'identity': u'2017075002312'}
{u'my_id': u'1883', u'identity': u'85500397082900013318629'}
{u'my_id': u'1698', u'identity': u'2017075002312'}
{u'my_id': u'1883', u'identity': u'85500397082900013318629'}
{u'my_id': u'1698', u'identity': u'2017075002312'}
{u'my_id': u'1883', u'identity': u'85500397082900013318629'}

And it keeps looping trough those two pairs.

EDITED:

I added the method do_scan.

Upvotes: 0

Views: 2201

Answers (1)

LexMulier
LexMulier

Reputation: 273

I'm not sure what your do_scan() does exactly, but this works:

def dynamo_scan(table_name, params_dict):
    dynamo_table = boto3.resource('dynamodb').Table(table_name)
    response = dynamo_table.scan(params_dict)
    results = response['Items']

    while 'LastEvaluatedKey' in response:
        response = dynamo_table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])
        results += response['Items']

    return results

Upvotes: 2

Related Questions