Delete batch operation in Azure Storage

Question

I have been trying to implement a DAO method for delete operation for Azure Storage entities. Delete using TableOperation was ok.

TableOperation deleteEntity = TableOperation.delete(entity);

But when I tried it using Batch Operation, It was not supported.

Any suggestions to overcome this issue is highly appreciated.

Answer 1

No, That was the code without using block operation. Following is the code that includes block operation. Sorry for not mentioning that

 TableBatchOperation batchOperation = new TableBatchOperation();
    List<TableBatchOperation> list = new ArrayList<>();

    if (partitionQuery != null) {
        for (AzureLocationData entity : cloudTable.execute(partitionQuery)) {

            batchOperation.add(TableOperation.delete(entity));
            list.add(batchOperation);    //exception thrown line
        }
        try {
            cloudTable.execute((TableOperation) batchOperation);
        } catch (StorageException e) {
            e.printStackTrace();
            }
            }

Answer 2

public void deleteLocationsForDevice(String id) {
    logger.info("Going to delete location data for Device [{}]", id);

    // Create a filter condition where the partition key is deviceId.
    String partitionFilter = TableQuery.generateFilterCondition(
            PARTITION_KEY,
            TableQuery.QueryComparisons.EQUAL,
            id);

    // Specify a partition query, using  partition key filter.
    TableQuery<AzureLocationData> partitionQuery =
            TableQuery.from(AzureLocationData.class)
                    .where(partitionFilter);



    if (partitionQuery != null) {
        for (AzureLocationData entity : cloudTable.execute(partitionQuery)) {

            TableOperation deleteEntity = TableOperation.delete(entity);
            try {
                cloudTable.execute(deleteEntity);
                logger.info("Successfully deleted location records with : " + entity.getPartitionKey());
            } catch (StorageException e) {
                e.printStackTrace();
            }

        }

    } else {
        logger.debug("No records to delete!");
    }

    //  throw new UnsupportedOperationException("AzureIotLocationDataDao Delete Operation not supported");
}

Answer 3

But when I tried it using Batch Operation, It was not supported.

I assumed that you could group your items for deleting by partition key, then execute the TableBatchOperation.

Here I wrote a helper class via C# language for achieving this purpose, you could refer to it:

public class TableBatchHelper<T> where T : ITableEntity
{
    const int batchMaxSize = 100;

    public static IEnumerable<TableBatchOperation> GetBatchesForDelete(IEnumerable<T> items)
    {
        var list = new List<TableBatchOperation>();
        var partitionGroups = items.GroupBy(arg => arg.PartitionKey).ToArray();
        foreach (var group in partitionGroups)
        {
            T[] groupList = group.ToArray();
            int offSet = batchMaxSize;
            T[] entities = groupList.Take(offSet).ToArray();
            while (entities.Any())
            {
                var tableBatchOperation = new TableBatchOperation();
                foreach (var entity in entities)
                {
                    tableBatchOperation.Add(TableOperation.Delete(entity));
                }
                list.Add(tableBatchOperation);
                entities = groupList.Skip(offSet).Take(batchMaxSize).ToArray();
                offSet += batchMaxSize;
            }
        }
        return list;
    }

    public static async Task BatchDeleteAsync(CloudTable table, IEnumerable<T> items)
    {
        var batches = GetBatchesForDelete(items);
        await Task.WhenAll(batches.Select(table.ExecuteBatchAsync));
    }
}

Then, you could you execute the batch deleting as follows:

await TableBatchHelper<ClassName>.BatchDeleteAsync(cloudTable,items);

Or

var batches = TableBatchHelper<ClassName>.GetBatchesForDelete(entities);
Parallel.ForEach(batches, new ParallelOptions()
{
    MaxDegreeOfParallelism = 5
}, (batchOperation) =>
    {
        try
        {
            table.ExecuteBatch(batchOperation);
            Console.WriteLine("Writing {0} records", batchOperation.Count);
        }
        catch (Exception ex)
        {
            Console.WriteLine("ExecuteBatch throw a exception:" + ex.Message);
        }
    });