CODEWITHSUNDEEP
azure-application-insightsazure-log-analyticsazure-data-explorer
ThomasC
ThomasC

Reputation: 118

Reference external data source from AI/Kusto query?

tl;dr: I want to reference an external data source from a Kusto query in Application Insights.

My application is writing logs to Application Insights, and we're querying it using Kusto in the Azure portal. To give an example of what I'm trying to do:

We're currently looking at these logs to find an action that triggers when a visitor viewed a blog post on our site. This is working well on a per blog-post level, but now we want to group this data by the category these blog posts are in, or by the tags they have, but that's not information I have within the logs.

The information we log contains unique info about that blog post (unique url, our internal id, etc) that I could use to look up this information in another data source (e.g. our SQL DB where this relation is stored), but I have no idea if/how this is possible. So that's the question, is this possible? Can I query a SQL DB, or get data in JSON via a URL or something?

Alternative solutions would be to move the reporting elsewhere (e.g. PowerBI) and just use AI as a data source, or to actually log all the category/tag info, but I really don't want to go down that route.

Upvotes: 5

Views: 1779

Answers (1)

Alexander Sloutsky
Alexander Sloutsky

Reputation: 3017

Kusto supports accessing external data (blobs, Azure SQL, Cosmos DB), however Application Insights / Azure Monitor and other multi-tenant services are blocking this functionality due to security and resource governance concerns.

You could try setting-up your own Azure Data Explorer (Kusto) cluster, where this functionality will be available, and then access your Application Insights data using cross-cluster query, or by exporting the data from Application Insights and hooking up EventGrid ingestion into your Kusto cluster.

Relevant links:

Kusto supporting external data: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/schema-entities/externaltables

Querying data inside Application Insights: https://learn.microsoft.com/en-us/azure/data-explorer/query-monitor-data

Continuous export data from Application Insights: https://learn.microsoft.com/en-us/azure/azure-monitor/app/export-telemetry

Data ingestion into Kusto from EventGrid: https://learn.microsoft.com/en-us/azure/data-explorer/ingest-data-event-grid

Upvotes: 5

Related Questions