CosmosDB : How to apply concurrency while inserting a document (in parallel requests)

Question

Background:

We have a EventHub where thousands of events are logged every day. The Azure function are configured on trigger over this eventhub on arrival of new messages. The azure function does following two tasks:

1. Write the raw message into document DB (collection 1)
2. Upsert an summary (aggregated) message into collection 2 of document Db. Before writing a message it checks if a summary message is already exists based on partition key and unique id (not id), it a doc exists then it update the doc with new aggregated value and if not then insert a new doc. This unique id is created based on a business logic.

Problem Statement:

More than one summary document is getting created for a PartitionKey and unique Id

Scenario Details

• let us say, for PartitionKey PartitionKey1 there is no summary document created in Collection for computed unique key.
• multiple messages (suppose 2) arrived at eventhub and which have triggered azure functions.
• all these 2 requests run concurrently, Since no existing document is found using the query, so each request make a message, now the Upsert function is invoked almost at the same time for writing summary document by concurrent request and resulted to have multiple summary documents for a PartitionKey and unique Id.

I've searched and read about Optimistic Concurrency which definitely I will implement for update scenario. but I could not able to find any way through which insert scenarios can be handled?

Answer 1

According to your description, I suggest you use Stored Procedure to achieve this.

Cosmos DB to guarantee ACID for all operations that are part of a single stored procedure.

As the official said: If the collection the stored procedure is registered against is a single-partition collection, then the transaction is scoped to all the documents within the collection. If the collection is partitioned, then stored procedures are executed in the transaction scope of a single partition key. Each stored procedure execution must then include a partition key value corresponding to the scope the transaction must run under.

For more information about Stored Procedure of Cosmos DB and how to create Stored Procedure, we can refer to:

Azure Cosmos DB server-side programming: Stored procedures, database triggers, and UDFs

Create and use stored procedures using C#