CODEWITHSUNDEEP
c#.netazure-table-storage
Alex Gordon
Alex Gordon

Reputation: 60691

How to retrieve more than 1000 results from table storage?

How can we retrieve more 1000 results from table storage?

Currently, this method is not retrieving more than 1000 results:

public static List<Translation> Get<T>(string sourceParty, string destinationParty, string wildcardSourceParty, string tableName) where T : ITableEntity, new()
{
    var acc = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable("RosettaCacheStorageConnection"));
    var tableClient = acc.CreateCloudTableClient();
    var table = tableClient.GetTableReference(Environment.GetEnvironmentVariable("RosettaTableCache"));

    TableQuery<T> treanslationsQuery = new TableQuery<T>().Where(
        TableQuery.CombineFilters(
            TableQuery.CombineFilters(
                TableQuery.GenerateFilterCondition("sourceParty", QueryComparisons.Equal, sourceParty.ToLowerTrim()),
                TableOperators.And,
                TableQuery.GenerateFilterCondition("destinationParty", QueryComparisons.Equal, destinationParty.ToLowerTrim())
                ), TableOperators.Or,
            TableQuery.CombineFilters(
                TableQuery.GenerateFilterCondition("sourceParty", QueryComparisons.Equal, wildcardSourceParty),
                TableOperators.And,
                TableQuery.GenerateFilterCondition("destinationParty", QueryComparisons.Equal, destinationParty.ToLowerTrim()))
        ));

    TableContinuationToken continuationToken = null;
    Task<TableQuerySegment<T>> translations = table.ExecuteQuerySegmentedAsync(treanslationsQuery, continuationToken);
    TableQuerySegment<T> translationss = translations.Result;

    List<Translation> trans = translationss.Cast<Translation>().ToList();

    return trans.Where(x => x.expireAt > DateTime.Now)
                       .Where(x => x.effectiveAt < DateTime.Now)
                       .ToList();
}

Is there a way to retrieve more 1000 results?

Please keep in mind that this is Azure Table Storage, and not cosmosdb.

Upvotes: 1

Views: 1926

Answers (2)

Daniel
Daniel

Reputation: 1016

By default Azure Table Storage will only return 1000 records at a given time, since you are calling ExecuteQuerySegmentedAsync() you are choosing to take 1000 records and use the continuationToken to fetch next set of records. Instead you can use ExecuteQuery(query); which will return all records expected. If you look at the outbound requests you will see that ExecuteQuery is handling the continuations for you in the library by making the multiple requests until all records are returned.

Change this:

Task<TableQuerySegment<T>> translations = table.ExecuteQuerySegmentedAsync(treanslationsQuery, continuationToken);

To this:

var translations = table.ExecuteQueryAsync(treanslationsQuery);

Upvotes: 2

Jon Skeet
Jon Skeet

Reputation: 1499770

I haven't used Azure Table Storage (at least not for a long time) but I suspect the problem is that you're only making a single call, passing in a null continuation token, and ignoring the continuation token in the result.

This code (which would be better off in an async method, awaiting the task) needs to be looped in order to get all the results:

TableContinuationToken continuationToken = null;
Task<TableQuerySegment<T>> translations = table.ExecuteQuerySegmentedAsync(treanslationsQuery, continuationToken);
TableQuerySegment<T> translationss = translations.Result;

I'd expect that you'd need to loop round, passing in the ContinuationToken property from the previous result into the ExecuteQuerySegmentedAsync method, until the continuation token is null to indicate the end of the results. So something like:

var results = new List<T>();
TableContinuationToken continuationToken = null;
do
{
    var response = await table.ExecuteQuerySegmentedAsync(treanslationsQuery, continuationToken);
    continuationToken = response.ContinuationToken;
    results.AddRange(response.Results);
} while (continuationToken != null);

Upvotes: 5

Related Questions