CODEWITHSUNDEEP
azure-data-explorer
Subodh Nijsure
Subodh Nijsure

Reputation: 3453

How to exclude "outlier" data in kusto

I am trying to track some network query responses.

In some cases, data measured is way out of the norm "5x" the normal data. Maybe developer was debugging something, but that data is still getting logged, etc.

Is there a way to exclude outlier data points in Kusto? So they don't impact the p95, p5 numbers?

Upvotes: 1

Views: 1158

Answers (1)

Casper Dijkstra
Casper Dijkstra

Reputation: 1885

This can be achieved by checking the value of the 95th percentile of your metric in TableX

let Percentile95 = toscalar(
    TableX
    | summarize percentile(metric, 95));

leading to something like enter image description here. Then you can use this number to filter out the outliers!

TableX
| where metric <= Percentile95

To filter out both 5% statistical outliers we would therefore use

let Percentile05 = toscalar(
    TableX
    | summarize percentile(metric, 5));
let Percentile95 = toscalar(
    TableX
    | summarize percentile(metric, 95));
TableX
| where metric <= Percentile95 and metric >= Percentile5

Upvotes: 2

Related Questions