CODEWITHSUNDEEP
azureazure-data-explorer
George Polevoy
George Polevoy

Reputation: 7671

How to ingest blobs created by Azure Diagnostics into Azure Data Explorer by subscribing to Event Grid notifications

I want to send Azure Diagnostics to Kusto tables.

The idea is to get logs and metrics from various Azure resources by sending them to a storage account.

I'm following both Ingest blobs into Azure Data Explorer by subscribing to Event Grid notifications and Tutorial: Ingest and query monitoring data in Azure Data Explorer, trying to use the best of all worlds - cheap intermediate storage for logs, and using EventHub only for notifications about the new blobs.

The problem is that only part of the data is being ingested. I'm thinking that the problem is in the append blobs which monitoring creates. When Kusto receives "Created" notification, only a part of the blob is written, and the rest of events are never ingested as the blob is appended to.

My question is, how to make this scenario work? Is it possible at all, or I should stick with sending logs to EventHub without using the blobs with Event Grid?

Upvotes: 1

Views: 280

Answers (1)

Vladik Branevich
Vladik Branevich

Reputation: 1175

Append blobs do not work nicely with Event Grid ADX ingestion, as they generate multiple BlobCreated events. If you are able to cause blob rename on update completion, that would sole the problem.

Upvotes: 1

Related Questions