CODEWITHSUNDEEP
azure-functionsazure-blob-storageazure-webjobssdkazure-eventgrid
sanjay kulkarni
sanjay kulkarni

Reputation: 197

Event Grid Trigger Azure Function: How to access metadata(user defined key, value pair) of deleted blobs?

How to get metadata of deleted blob in Event Grid Trigger Azure Function? Below is a sample C# code -

[FunctionName("EventGridTriggerFunction")] 
public static void Run([EventGridTrigger]EventGridEvent eventGridEvent, ILogger log) 
{ log.LogInformation(eventGridEvent.Data.ToString());
}

Can I get from EventGridEvent --> data object property? Is there any way to set custom event schema with blob's metadata?

Referring link - https://learn.microsoft.com/en-us/azure/event-grid/event-schema-blob-storage

[{
  "topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account",
  "subject": "/blobServices/default/containers/test-container/blobs/new-file.txt",
  "eventType": "Microsoft.Storage.BlobCreated",
  "eventTime": "2017-06-26T18:41:00.9584103Z",
  "id": "831e1650-001e-001b-66ab-eeb76e069631",
  "***data***": {
    "api": "PutBlockList",
    "clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760",
    "requestId": "831e1650-001e-001b-66ab-eeb76e000000",
    "eTag": "\"0x8D4BCC2E4835CD0\"",
    "contentType": "text/plain",
    "contentLength": 524288,
    "blobType": "BlockBlob",
    "url": "https://my-storage-account.blob.core.windows.net/testcontainer/new-file.txt",
    "sequencer": "00000000000004420000000000028963",
    "storageDiagnostics": {
      "batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
    }
  },
  "dataVersion": "",
  "metadataVersion": "1"
}]

Upvotes: 1

Views: 2275

Answers (2)

warghs
warghs

Reputation: 39

Thank you @Roman Kiss for this. This workaround is so helpful to get out of the infinite Delete Blob loop when undeleting/re-deleting in the event handler.

To add to the solution, if you are using the Azure python sdk, here's how you set the content type when deleting a blob as it's not completely obvious in the api:

blob_service_client = BlobServiceClient.from_connection_string(
      conn_str=os.getenv('AZURE_STORAGE_CONNECTION_STRING'))
blob_client = blob_service_client.get_blob_client(
      container = container_name,
      blob = blob_name)
blob_client.set_http_headers(content_settings=ContentSettings(content_type=CUSTOM_CONTENT_TYPE))
blob_client.delete_blob()

Upvotes: 0

Roman Kiss
Roman Kiss

Reputation: 8265

There is no elegant workaround for your solution. However, turn-on soft delete for blobs option in the storage account will enable to get the blob metadata in the EventGridTrigger subscriber after calling the undelete blob request.

The following code snippet shows this implementation example:

run.csx:

#r "Newtonsoft.Json"
#r "Microsoft.WindowsAzure.Storage"

using System;
using System.Threading.Tasks;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Linq;


public static async Task Run(JObject eventGridEvent, CloudBlockBlob blob, ILogger log)
{
    log.LogInformation($"{eventGridEvent}");

    if (eventGridEvent["data"]["contentType"].Value<string>() != "abcd" && eventGridEvent["data"]["api"].Value<string>() == "DeleteBlob")
    {
        await blob.UndeleteAsync();
        await blob.FetchAttributesAsync();           
    
        log.LogInformation($"\nMetadata: {string.Join(" | ", blob.Metadata.Select(i => $"{i.Key}={i.Value}"))}");
    
        // ...
    
        blob.Properties.ContentType = "abcd";
        await blob.SetPropertiesAsync();
        await blob.DeleteAsync();
    }
    else if (eventGridEvent["data"]["contentType"].Value<string>() != "abcd")
    {
        await blob.FetchAttributesAsync();           
    
        log.LogInformation($"\nMetadata: {string.Join(" | ", blob.Metadata.Select(i => $"{i.Key}={i.Value}"))}");

    }

    await Task.CompletedTask;
}

function.json:

{
  "bindings": [
    {
      "type": "eventGridTrigger",
      "name": "eventGridEvent",
      "direction": "in"
    },
    {
      "type": "blob",
      "name": "blob",
      "path": "{data.url}",
      "connection": "rk2018ebstg_STORAGE",
      "direction": "in"
    }
  ],
  "disabled": false
}

Note, that the data object in the event message emitted by storage account included only two properties of the blob file such as contentType and contentLength.

The above implementation used for avoiding the cycling of the DeleteBlob by subscriber the blob property contentType with unexpected value, for instance: abcd.

It will be nice, if the soft deleted blob allows to retrieve its properties and/or metadata. I have wrote the feedback for this option here.

Upvotes: 3

Related Questions