CODEWITHSUNDEEP
prometheusalertgrafana
MattArgo
MattArgo

Reputation: 41

Create a Prometheus Query to Alert when A Value has Been Above Threshold for Interval

I'm trying to create a Prometheus query that will alert when the value of the metric is above a certain threshold for a given period of time.

In my case, I'd like the alert to fire if the value for the metric is over 0 for more than 10 minutes.

I thought about getting the average over that interval and if the average is over 0:

sum(sum_over_time(name_of_metric{job="name_of_release"}[10m])) / sum(count_over_time(name_of_metric{job="name_of_release"}[10m])) > 0

But that will not work since the average will always be above 0 if the metric is > 0 for any moment in that 10 minute period.

Is there any way I can tell if a metric has a sustained value over a period of time?

Upvotes: 2

Views: 3820

Answers (1)

valyala
valyala

Reputation: 17794

Try the following query:

name_of_metric{job="name_of_release"}
  unless
(min_over_time(name_of_metric{job="name_of_release"}[10m]) <= 0)

It will trigger for name_of_metric{job="name_of_release"} time series if all its samples are bigger than 0 for the last 10 minutes.

See docs for min_over_time function and docs for unless operator.

Upvotes: 4

Related Questions