CosmosDB CreateDocumentQuery with EnableCrossPartitionQuery does not return results but does if specifying the PartitionKey

Question

I have a problem where, if I specify the Partition Key for a specific query, I will get the record that I expect. However, if I do not specify a Partition Key and just set EnableCrossPartitionQuery to true, it will not return/find any documents.

This actually works as expected on one of my Cosmos DBs but not on another one. Same records/documents.

I'm using the following setup

1. Microsoft.Azure.DocumentDB NuGet package version 2.3.0

2. Cosmos DB Collection with Unlimited capacity (with PartitionKey = ApplicationId)

3. Three very simple documents on the DB/Collection, created manually through Azure Storage Explorer

My code looks like below, fairly simple. If the caller of GetDocuments passes a value for partitionKey, then I specify it on the query through FeedOptions. Otherwise, I set EnableCrossPartitionQuery on the FeedOptions.

The documents on the Database/Collection that does work are the same as the ones on the DB/Collection that does not work.

I created the Collections in the same way, with the same Partition Key (ApplicationId)

public async Task<IEnumerable<T>> GetDocuments<T>(Expression<Func<T, bool>> predicate, object partitionKey = null)
{
    IDocumentQuery<T> queryDetails = QueryDocument<T>(predicate, partitionKey);

    var queryData = await queryDetails.ExecuteNextAsync<T>();

    if (queryData.Any())
    {
        return queryData;
    }

    return default(IEnumerable<T>);
}

private IDocumentQuery<T> QueryDocument<T>(Expression<Func<T, bool>> predicate, object partitionKey = null)
{
    FeedOptions feedOptions;

    if (partitionKey == null)
    {
        feedOptions = new FeedOptions { EnableCrossPartitionQuery = true };
    }
    else
    {
        feedOptions = new FeedOptions { PartitionKey = new PartitionKey(partitionKey) };
    }

    var query = _client.CreateDocumentQuery<T>(DocumentCollectionUri, feedOptions);

    var queryDetails = query.Where(predicate).AsDocumentQuery();

    return queryDetails;
}

The document looks like this:

{
"id": "1",
"HubName": "abxyz-hub",
"ClientId": "abxyz",
"ApplicationId": 1,
"ApplicationName": "My App Name",
"_rid": "hSkpAJde99IBAAAAAAAAAA==",
"_self": "dbs/hSkpAA==/colls/hSkpAJde99I=/docs/hSkpAJde99IBAAAAAAAAAA==/",
"_etag": "\"53007677-0000-0100-0000-5cbb3c660000\"",
"_attachments": "attachments/",
"_ts": 1555774566
}

Any ideas why this does not work?

Answer 1

Your code is wrong. Your mistake is in this check:

if (queryData.Any())

You are returning before you actually get all the data back.

The reason why your code works with a partition key is because you are targeting one physical partition (via a logical partition) and the data contained is less than the MaxItemCount or the RequestOptions object you provide.

Cosmos DB only returns paginated results, and it needs to call every physical partition for those values, some times multiples times each partition and in some cases an iteration might return 0 data but the next one might have some. You must ExecuteNextAsync until HasMoreResults is false.

Adding a while loop to get all the paged results from all the physical partitions that your cross partition query will hit will solve the problem:

Here is the code:

public async Task<IEnumerable<T>> GetDocuments<T>(Expression<Func<T, bool>> predicate, object partitionKey = null)
{
    IDocumentQuery<T> documentQuery = QueryDocument<T>(predicate, partitionKey);

    var results = new List<T>();

    while(documentQuery.HasMoreResults)
    {
        var docs = await documentQuery.ExecuteNextAsync<T>();    
        results.AddRange(docs)
    }

    return results;
}

private IDocumentQuery<T> QueryDocument<T>(Expression<Func<T, bool>> predicate, object partitionKey = null)
{
    FeedOptions feedOptions;

    if (partitionKey == null)
    {
        feedOptions = new FeedOptions { EnableCrossPartitionQuery = true };
    }
    else
    {
        feedOptions = new FeedOptions { PartitionKey = new PartitionKey(partitionKey) };
    }

    var query = _client.CreateDocumentQuery<T>(DocumentCollectionUri, feedOptions);

    var queryDetails = query.Where(predicate).AsDocumentQuery();

    return queryDetails;
}