CODEWITHSUNDEEP
c#entity-frameworkazureazure-sql-databaseazure-functions
Mandar Jogalekar
Mandar Jogalekar

Reputation: 3281

Entity Framework Concurrency and Azure functions

This question may not be only specific to Azure functions, but for me it is the scenario. I have azure function which is processing messages from service bus topic subscription. Messages are for adding/updating data in the table on sql azure server.

For Database operations, I am using Entity framework 6.2

My messages can be 100's together, but it can be that only first message is the one that should insert entry in the database, and the next few are updates of same record.

Now, In my Azure functions, My host.json looks like this.

{
 "serviceBus": {
 "maxConcurrentCalls": 16,
 "prefetchCount": 20,
 "autoRenewTimeout": "00:05:00"
}
}

which means, it should be able to handle 16 messages concurrently, which it does.

however, with concurrent processing, entity framework context is not updated from the parallel running transaction, and I end up having 10 rows where I should have only 1 row in the database, as it's essentially the same record.

Is there a entity framework specific solution to this?

Upvotes: 1

Views: 1016

Answers (1)

Mikhail Shilkov
Mikhail Shilkov

Reputation: 35134

What you want to do is to limit the processing of messages belonging to the same entity to 1 at a time. There are several approaches to do that:

  1. To solve it on Entity Framework level, you would need to correctly wrap your read-write operations into a single transaction with Serializable isolation level. However it doesn't really make much sense: you do parallel processing of the messages, but limit the parallelization with expensive locks on database level.

  2. Use database-level constraints, like unique index, and then make sure that each message inserts a row with the same unique key for the same entity. The second operation which tries to write it will get a constraint violation exception, will retry the operation which will lead to update.

  3. Just limit maxConcurrentCalls to 1 and never run more than a single instance of your Function. That's the easiest way, but obviously it doesn't scale for higher loads.

  4. Use Service Bus sessions based on entity key, i.e. send messages for the same entities to the same session. Bad news though: Service Bus sessions are not supported by Azure Functions, so you would have to fall back to manual processing.

  5. Use e.g. Event Hubs instead of Service Bus subscription. Partition events by entity ID and then configure your Function as described in In order event processing with Azure Functions.

Upvotes: 1

Related Questions