CODEWITHSUNDEEP
azureazure-cosmosdb
Vanathi
Vanathi

Reputation: 195

Change Feed Processor Lib does not honour ChangeFeedProcessorOptions FeedPollDelay / CheckPointFrequency

I am following this sample code (https://github.com/Azure/azure-documentdb-changefeedprocessor-dotnet#example) to register an observer to process change feed in cosmos db collection. I am creating new documents in the cosmos db collection using a utility (say create 400 documents within a for loop). I am using using FeedPollDelay of 30 seconds. But it doesn't seem to be honoured by CFP lib. ProcessChangesAsync method gets invoked repeatedly even before feed poll delay interval expires. In the first batch, around 60 docs are retrieved and in the second batch around 20 docs are retrieved, in the third batch around 100 docs are retrieved.

        DocumentCollectionInfo feedCollectionInfo = new DocumentCollectionInfo()
        {
            DatabaseName = databaseName,
            CollectionName = monitoredCollectionName,
            Uri = new Uri(uri),
            MasterKey = masterKey
        };

        DocumentCollectionInfo leaseCollectionInfo = new DocumentCollectionInfo()
        {
            DatabaseName = databaseName,
            CollectionName = leaseCollectionName,
            Uri = new Uri(uri),
            MasterKey = masterKey
        };

        ChangeFeedProcessorOptions feedProcessorOptions = new ChangeFeedProcessorOptions()
        {
            FeedPollDelay = TimeSpan.FromSeconds(30)
            //LeasePrefix = Guid.NewGuid().ToString(),
            //MaxItemCount = 100
        };
        ChangeFeedProcessorBuilder builder = new ChangeFeedProcessorBuilder();
        processor = await builder
            .WithHostName(hostName)
            .WithFeedCollection(feedCollectionInfo)
            .WithLeaseCollection(leaseCollectionInfo)
            .WithProcessorOptions(feedProcessorOptions)
            .WithObserver<LiveWorkItemChangeFeedObserver>()
            .BuildAsync();

        await processor.StartAsync();

Receiving 60 docs in first batch is fine. But I am expecting the second batch to be invoked with rest 340 docs in a single batch after the feed poll delay (30 seconds) interval expires.

But ProcessChangesAsync method gets triggered frequently and this option is not being honoured.

Upvotes: 1

Views: 514

Answers (1)

Matias Quaranta
Matias Quaranta

Reputation: 15603

FeedPollDelay is used when the Change Feed Processor reads the Change Feed and finds no new changes, not in-between each batch.

Example flow:

  1. CFP polls for changes, finds X.
  2. ProcessChangesAsync is called with X
  3. After ProcessChangesAsync finishes, CFP immediately polls for changes, finds Y.
  4. ProcessChangesAsync is called with Y.
  5. After ProcessChangesAsync finishes, CFP immediately polls for changes, finds nothing, waits FeedPollDelay.
  6. CFP polls for changes, finds Z.
  7. ProcessChangesAsync is called with Z
  8. After ProcessChangesAsync finishes, CFP immediately polls for changes, finds nothing, waits FeedPollDelay.
  9. Etc….

Upvotes: 1

Related Questions