Dhody Rahmad Hidayat
Dhody Rahmad Hidayat

Reputation: 457

Grafana alert from percentage CPU/Memory usage

I want to make an alert through Grafana that define if the CPU or Memory usage above threshold (let say 85%) it will firing an alert. What I have now are time series limit CPU/memory

kube_pod_container_resource_limits{namespace="$namespace", pod="$pod", resource="cpu"}

and pod resource usage

sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod="$pod", container!="POD", container!="", pod!=""}[1m]))

Logical to make the percentage is, (resource_usage_query)/(resource_limit_query)*100. I've tried to combine both query same as the formula but ended with nodata value. Is there any syntax or something I missed? I appreciate any suggestion.

Upvotes: 2

Views: 7990

Answers (1)

Sergio Santiago
Sergio Santiago

Reputation: 1514

You are ending with no data because the metrics have different labels. You need to aggregate both by e g: pod , then do the division. This should fix your problem. Something like: (I didn't test it)

sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod="$pod", container!="POD", container!="", pod!=""}[1m])) by (pod) / sum(kube_pod_container_resource_limits{namespace="$namespace", pod="$pod", resource="cpu"}) by (pod) * 100

Upvotes: 2

Related Questions