Paging with CosmosClient in CosmosDB

Question

I'm trying to implement paging using the SDK v3 CosmosClient instead of the old DocumentClient.

Reason for this is it seems DocumentClient doesn't translate LINQ queries that contains spatial functions very well (ie: When using Within() I'll get an error from DocumentClient stating the methods is not implemented).

Paging works well with DocumentClient.CreateDocumentQuery<T> as such:

var query = DocumentClient.CreateDocumentQuery<T>(UriFactory.CreateDocumentCollectionUri("master", "features"), feedOptions)
                .Where(t => t.Type == typeof(T).Name)
                .Where(pred)
                .AsDocumentQuery();

string queryContinuationToken = null;
var page = await query.ExecuteNextAsync<T>();
if (query.HasMoreResults)
    queryContinuationToken = page.ResponseContinuation;

I'm at a little loss as to where to gather a continuation token using CosmosClient and its Container class:

QueryRequestOptions options = new QueryRequestOptions();
options.MaxItemCount = maxRecords;

FeedIterator<T> feed;

if (continuationToken == "")
    feed = Container.GetItemLinqQueryable<T>(true, null, options).Where(x => x.Type == typeof(T).Name).Where(pred).ToFeedIterator();
else
    feed = Container.GetItemLinqQueryable<T>(true, continuationToken, options).Where(x => x.Type == typeof(T).Name).Where(pred).ToFeedIterator();

FeedIterator seems to have some of the members IDocumentQuery has (like HasMoreResults) but I can't find a continuation token anywhere.

What am I missing?

Answer 1

IQueryable<returnVModel> query;
var requestOptions = new QueryRequestOptions
{
 MaxItemCount = 20
};
if (Token == "" || Token == null)
{
      query = Container.GetItemLinqQueryable<returnVModel>(false, null, requestOptions).Where(x => x.id == id);
 }
 else
 {
      query = Container.GetItemLinqQueryable<returnVModel>(false, Token, requestOptions).Where(x => x.id == id);
 }
 var ct = new CancellationTokenSource();
 var totalCount = await query.CountAsync(ct.Token); //Total Count
 var feedIterator = query.ToFeedIterator();
 var queryResults = new List<returnVModel>();
 FeedResponse<returnVModel> feedResults = await feedIterator.ReadNextAsync(ct.Token);
 queryResults.AddRange(feedResults); // Output
 var PaginationToken = feedResults.ContinuationToken //Token

First time we need to pass token as null, from next page onwards pass the token which we received in previous output.

Pagination was working fine in v3.

Answer 2

With the version 3.12.0 of the Cosmos SDK the following works as expected as a pretty much in place replacement for the older DocumentQuery.

Original DocumentClient method for comparison:

IDocumentQuery<ToDoItem> query = client.CreateDocumentQuery<ToDoItem>(collectionUri)
            .Where(t => t.Description.Contains(searchterm))
            .AsDocumentQuery();

while (query.HasMoreResults)
{
  foreach (ToDoItem result in await query.ExecuteNextAsync())
  {
    log.LogInformation(result.Description);
  }
}

Using a CosmosClient this becomes:

var database = client.GetDatabase("ToDoItems");
var container = database.GetContainer("Items");

var query = container.GetItemLinqQueryable<ToDoItem>()
     .Where(t => t.Description.Contains(searchTerm))
     .ToFeedIterator();

while (query.HasMoreResults)
{
  foreach (ToDoItem result in await query.ReadNextAsync())
  {
    log.LogInformation(result.Description);
  }    
}

So your query is now a FeedIterator, and you can call HasMoreResults and ReadNextAsync on it.

Admittedly this won't get you access to the diagnostics, request charge, etc. that comes on the FeedIterator, but it will page through the results cleanly.

Answer 3

Alright, here's a Where method I implemented. Seems to work at first glance.

If you do var f = feed.ReadNextAsync() you won't get an object that's of type FeedResponse, preventing you access to the token. You need to declare f explicitly of type FeedResponse<T>

public async Task<(IEnumerable<T> Results, string ContinuationToken)> Where<T>(Expression<Func<T, bool>> pred, int maxRecords = 0, string partitionKey = "", string continuationToken = "") where T : IDocumentModel
{

    QueryRequestOptions options = new QueryRequestOptions();

    if (partitionKey != "")
        options.PartitionKey = new PartitionKey(partitionKey);


    if (maxRecords == 0)
    {
        return (Container.GetItemLinqQueryable<T>(true, null, options).Where(x => x.Type == typeof(T).Name).Where(pred), "");
    }
    else
    {
        options.MaxItemCount = maxRecords;
        string token = "";
        FeedIterator<T> feed;
        List<T> res = new List<T>();

        if (continuationToken == "")
            feed = Container.GetItemLinqQueryable<T>(true, null, options).Where(x => x.Type == typeof(T).Name).Where(pred).ToFeedIterator();
        else
            feed = Container.GetItemLinqQueryable<T>(true, continuationToken, options).Where(x => x.Type == typeof(T).Name).Where(pred).ToFeedIterator();

        Microsoft.Azure.Cosmos.FeedResponse<T> f = await feed.ReadNextAsync();
        token = f.ContinuationToken;

        foreach (var item in f)
        {
            res.Add(item);
        }

        return (res, token);
    }

}