CODEWITHSUNDEEP
c#azure-table-storageazure-functions
Steph Locke
Steph Locke

Reputation: 6166

Getting "Entity already exists" error writing aggregates to Azure Table Storage (with Azure Function)

In an Azure Function, I'm trying to aggregate some data and write the aggregations into a Table.

I have a query that summarises data:

var query = recs
            .GroupBy(r => new { r.Category, r.Account, r.Record })
            .Select(r => new ts_webhitaggregate_account
                    {
                        PartitionKey = partition,  // Constant
                        RowKey = $"{r.Key.Category}:{r.Key.Account}:{r.Key.Record}", // Group By 
                        rawDate = intervaldate,   // Constant
                        epochDate = intervalepoch, // Constant
                        Category = r.Key.Category, // Group By 
                        Account = r.Key.Account, // Group By 
                        Record = r.Key.Record, // Group By 
                        Hits = r.Count(), // Aggregate
                        Users = r.Select(t => t.User).Distinct().Count(), // Aggregate
                        Devices = r.Select(t => t.Device).Distinct().Count() // Aggregate
                    });

I then attempt to pass these records to the ICollector bound Table

foreach (ts_webhitaggregate_account a in query.ToList())
{
    webhitsAggAccount.Add(a);
}

I'm frequently encountering an "Entity already exists" error like:

Exception has been thrown by the target of an invocation. Microsoft.WindowsAzure.Storage: 82:The specified entity already exists.

If I was writing a comparable SQL statement to the C# I wouldn't expect duplicates of the compound key to be possible as every value being written is the key, a constant, or an aggregation. I also have no pre-existing data in the Table which could be causing the conflict.

What am I doing wrong to be generating duplicates in my query?

Upvotes: 2

Views: 6902

Answers (4)

user160357
user160357

Reputation: 1526

Use "TableClient.UpsertEntity" : create if not exist and replace if exist

CallStatusEntity customerEntity = new CallStatusEntity()
{
    PartitionKey = callerId,
    RowKey = "",
    IsRunning = true
};

tableClient.UpsertEntity(customerEntity);

MS Docs

Upvotes: 0

Steph Locke
Steph Locke

Reputation: 6166

I believe I found where I was being stupid ... these happen inside a loop for the category, and I should have been selecting a single category each time, but one range selection on the row key was including another category that then went on to be selected, twice inserted to the second category.

When in doubt, print everything to the console!

Upvotes: 1

Dogu Arslan
Dogu Arslan

Reputation: 3383

Just a side comment and may be helpful for you to debug. In the error message you pasted "82:The specified entity already exists." number 82 is the index of the problematic (duplicate) item in the batch operation.

Upvotes: 0

Jambor - MSFT
Jambor - MSFT

Reputation: 3293

The error message told us that an entity that already exists. As I know, the collection seems has an entity with the same PartitionKey and RowKey. Please try to use insert or update method to see whether it could give you help.

Upvotes: 0

Related Questions