Reputation: 457
I want to make an alert through Grafana that define if the CPU or Memory usage above threshold (let say 85%) it will firing an alert. What I have now are time series limit CPU/memory
kube_pod_container_resource_limits{namespace="$namespace", pod="$pod", resource="cpu"}
and pod resource usage
sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod="$pod", container!="POD", container!="", pod!=""}[1m]))
Logical to make the percentage is, (resource_usage_query)/(resource_limit_query)*100
. I've tried to combine both query same as the formula but ended with nodata
value. Is there any syntax or something I missed? I appreciate any suggestion.
Upvotes: 2
Views: 7990
Reputation: 1514
You are ending with no data because the metrics have different labels. You need to aggregate both by e g: pod , then do the division. This should fix your problem. Something like: (I didn't test it)
sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod="$pod", container!="POD", container!="", pod!=""}[1m])) by (pod) / sum(kube_pod_container_resource_limits{namespace="$namespace", pod="$pod", resource="cpu"}) by (pod) * 100
Upvotes: 2