vinoth
vinoth

Reputation: 305

Not able to get more than 1000 rows from azure storage table

I am beginner in Python and try to retrieve all rows (more than 1000 rows) from azure storage table. Below is the sample code. The code gives me 1000 records but the table (testtable) has more than 50000 rows.i read in some blog using continuation token can pull all records. Let me know how can i implement in this

table='testtable'
now2='14042018'
count=0

try:
  table_service = TableService(account_name=myaccount, account_key=mykey)
  logging.info('connected successfully')

except Exception as e:
  logging.error(e)



tasks = table_service.query_entities(table,filter='PartitionKey eq \'' + now2 + '\'')

for task in tasks:
  count=count+1
  a=task.desc
  #print(a)
print(count)

Upvotes: 0

Views: 414

Answers (1)

Ivan Glasenberg
Ivan Glasenberg

Reputation: 29995

Update:

Even if I just use this line of code:

entities = table_service.query_entities(table,filter='PartitionKey eq \'' + now2 + '\'')

and all of the rows in my table can be fetched(more than 10000 rows).


Use the code below:

from azure.cosmosdb.table.tableservice import TableService
from azure.cosmosdb.table.models import Entity

table_service = TableService(account_name='your account',account_key='your key')

table='tasktable'
now2='03042018'
count=0       

next_pk=None
next_rk = None

while True:
    entities = table_service.query_entities(table,filter='PartitionKey eq \'' + now2 + '\'')
    for entity in entities:
        count=count+1

    if hasattr(entities, 'x_ms_continuation'):
        x_ms_continuation = getattr(entities, 'x_ms_continuation')
        next_pk = x_ms_continuation['nextpartitionkey']
        next_rk = x_ms_continuation['nextrowkey']
    else:
        break

print(count)

Test result as below: enter image description here

Upvotes: 1

Related Questions