CODEWITHSUNDEEP
azureazure-cosmosdbazure-functions
Metehan Mutlu
Metehan Mutlu

Reputation: 137

Azure function CosmosDbTrigger (Start from the beginning option)

I have a azure function with cosmos db trigger which makes some calculations and write results to db. If something goes wrong i want to have a possibility to start from the first item or specific item make calculations again. Is it possible? Thanks

public static void Run([CosmosDBTrigger(
        databaseName: "db",
        collectionName: "collection",
        ConnectionStringSetting = "DocDbConnStr",
        CreateLeaseCollectionIfNotExists = true,
        LeaseCollectionName = "leases")]IReadOnlyList<Document> input, TraceWriter log)
    {

        ...
    }

Upvotes: 3

Views: 2582

Answers (4)

vullnetyy
vullnetyy

Reputation: 1701

As @Matias Quaranta and @Pankaj Rawat say in the comments, the accept answer is old and is no longer true. You can use StartFromTheBeginning as a C# attribute within your azure function's parameter list like so:

[FunctionName(nameof(MyAzureFunction))]
public async Task RunAsync([CosmosDBTrigger(
    databaseName: "myCosmosDbName",
    collectionName: "myCollectionName",
    ConnectionStringSetting = "cosmosConnectionString",
    LeaseCollectionName = "leases",
    CreateLeaseCollectionIfNotExists = true,
    MaxItemsPerInvocation = 1000,
    StartFromBeginning = true)]IReadOnlyList<Document> documents)
{
    ....
}

Please change the accepted answer.

Upvotes: 3

Pankaj Rawat
Pankaj Rawat

Reputation: 4583

Check startFromBeginning option available in Function v2. Unfortunately, I'm still using V1 and not able to verify.

When set, it tells the Trigger to start reading changes from the beginning of the history of the collection instead of the current time. This only works the first time the Trigger starts, as in subsequent runs, the checkpoints are already stored. Setting this to true when there are leases already created has no effect.

Upvotes: 0

Matias Quaranta
Matias Quaranta

Reputation: 15603

Right now, the StartFromBeginning option is not exposed to the Cosmos DB Trigger. The default behavior is to start receiving changes from the moment the Function starts running, leases/checkpoints will be generated in case the Host/Runtime shutsdown so when the Host/Runtime is back up it will pickup from the last checkpointed item.

The Trigger does not implement dead-lettering or error handling as it might generate infinite-loops / unexpected billing / multiple processing of the same batch if the error is not related to the batch itself (for example, you process the documents and then send an email and the email fails, the entire batch would be re-processed for an error not related to the feed itself), so we recommend users to implement their own try/catch or error handling logic inside the Function's code. It's the same approach as the Event Hub Trigger.

That being said, we are in the process of exposing several new options on the Trigger and there is a contributor working on an advanced retrying mechanism.

Upvotes: 3

Mikhail Shilkov
Mikhail Shilkov

Reputation: 35144

The current offsets (positions in Cosmos DB change feed) are managed by clients, Azure Functions runtime in this case.

Functions store the offsets in lease collection (it's called leases in your example).

To restart from a specific item, you would have to make a snapshot of documents in leases collection at some point, and then restore your current collection to that snapshot when needed.

I am not familiar with a tool that automates that for you, other than generic tools working with Cosmos DB collections.

Upvotes: 1

Related Questions