CODEWITHSUNDEEP
prometheusprometheus-alertmanagerpromqlprometheus-operator
Santosh M.
Santosh M.

Reputation: 2454

Generate alert if percentage drop is more than 70% for metricX at any point of time for 5 mins

I would like to generate alert for example when counter metricX drops more than 70% at any point of time for 5 mins.

Following would be my rule YAML file to generate alert:

groups:
- name: MetricX dip
  rules:
  - alert: MetricX dip by more than 70%
    expr: 
    for: 0m
    labels:
      severity: warning
    annotations:
      descriptions: MetricX has been dropped by more than 70%

I would like to see some guide on how can I write promQL expressions in rule file to measure drop in percentage of metricX at any point of time for 5 minutes.

Upvotes: 3

Views: 2407

Answers (2)

valyala
valyala

Reputation: 17794

Try the following query:

rate(metricX[2m]) < 0.7 * (rate(metricX[2m] offset 5m))

Upvotes: 0

Santosh M.
Santosh M.

Reputation: 2454

After understanding promql throughly, this is what I implemented in simple form which satisfies my requirement.

(-100 * rate(MetricX[2m]) / rate(MetricX[5m] offset 1m)) > 70

MetricX is counter. So, I used rate in computing drop in percentage. My prometheus scrape_interval is 1 min. So, dip is computed over 2 mins that follows 5 mins.

I hope this helps other people.

Upvotes: 2

Related Questions