App Insights GetMetric("name").TrackValue doesn't aggregate when viewing in App Insights portal

Question

I am sending a custom metric to Application Insights (Azure Monitor?) using the following C# code in an Azure Function.

(it measures the elapsed time between Service Bus message enqueued and Function code execution)

this.TelemetryClient.GetMetric("QueueTriggerLagMs").TrackValue((now - serviceBusMessage.SystemProperties.EnqueuedTimeUtc).TotalMilliseconds);

I'd expect this to be aggregated by the SDK and shown as an aggregate in the portal but its summing it, I think.

Microsoft.ApplicationInsights.TelemetryClient.TrackMetric is not the preferred method for sending metrics. Metrics should always be pre-aggregated across a time period before being sent. Use one of the GetMetric(..) overloads to get a metric object for accessing SDK pre-aggregation capabilities. If you are implementing your own pre-aggregation logic, you can use the TrackMetric() method to send the resulting aggregates. If your application requires sending a separate telemetry item at every occasion without aggregation across time, you likely have a use case for event telemetry; see TelemetryClient.TrackEvent (Microsoft.ApplicationInsights.DataContracts.EventTelemetry).

Source https://learn.microsoft.com/en-gb/azure/azure-monitor/app/api-custom-events-metrics

It's funny, while typing this question out I re-read and noticed that the wording in that pullquote is confusing and that I need to do my own aggregation.

Use one of the GetMetric(..) overloads to get a metric object for accessing SDK pre-aggregation capabilities.

Makes it sound like by using that method instead of TrackMetric then it'll pre-aggregate but I don't think that's what they're saying.

I thought the built-in behaviour would be to aggregate for me, instead of forcing every developer to write the same code to aggregate over some batching time period that we don't know.

I'm completely lost.

Does anyone know what I'm supposed to do to track that lag time?

---

Relates to question.

Azure Function is not logging LogMetric telemetry to Application Insights

Answer 1

Before getting to metric details... The question implies that "sum" is not an "aggregate", but "aggregate" is a generic term for any value or process that represents many combined values in some manner. In other words, a "sum" is just one type of aggregate; "average" is another, etc. It's not clear exactly what you expect.

GetMetric() returns an object that automatically does local aggregation over 60 second intervals before sending details to the App Insight telemetry server, but it is special because although it aggregates some things like count, sums and averages it still allows alerts to be triggered from individual values (if configured).

The purpose of GetMetric() is to do the pre-aggregation for you to reduce traffic and telemetry storage and cost. The alternative is to call telemetryClient.TrackMetric() which will record each and every value, but at increased cost.

See GetMetric() documentation for details.

Lastly, it could be that you're visualizing the data incorrectly in App Insights. When tracking milliseconds (or any time value), I suggest looking at an Avg aggregation in a Scatter Plot graph. Don't use the Line or Area plots since they do a poor job of interpolating between missing points and paint a wildly-fluctuating curve.

Probably don't use the Count aggregation either since that won't tell you anything about the custom "elapsed timing" values, rather it will only tell you about how often the metric was sent. That might provide useful answers, but only for the right question.

Answer 2

TrackValue sums up provided values and also tracks valueCount. It allows to get average during query time:

Metrics blade also allows to pick Aggregation: