CODEWITHSUNDEEP
kubernetesgrafanaprometheuspromql
GihanS
GihanS

Reputation: 503

Prometheus queries to get CPU and Memory usage in kubernetes pods

I need to get CPU and Memory usage in kubernetes pods with prometheus queries. Can someone plz help?

Upvotes: 16

Views: 83779

Answers (4)

valyala
valyala

Reputation: 18056

The following query should return per-pod number of used CPU cores:

sum(rate(container_cpu_usage_seconds_total{container!=""}[5m])) without (container)

The following query should return per-pod RSS memory usage:

sum(container_memory_working_set_bytes{container!=""}) without (container)

See this answer about container!="" filter in queries above.

If you need summary CPU and memory usage across all the pods in Kubernetes cluster, then just remove without (container) suffix from queries above.

Upvotes: 7

zangw
zangw

Reputation: 48526

To better monitor the memory usage of Pod/Containers. The container_memory_max_usage_bytes should also be used to monitor besides container_memory_working_set_bytes.

We used to monitor the memory usage of the pod through

avg(container_memory_working_set_bytes{container!="POD"}) by (pod)  / (avg(kube_pod_container_resource_requests_memory_bytes{container!="POD"} > 0) by (pod)) *100

However, we met the OOMKilled of the pod and failed to find anything abnormal from the above metrics.

Through the container_memory_max_usage_bytes we find the abnormal memory usage of the pod.

avg(container_memory_max_usage_bytes{ container!="POD"}) by (pod)  / (avg(kube_pod_container_resource_requests_memory_bytes{ container!="POD"} > 0) by (pod)) *100

Upvotes: 1

Peter
Peter

Reputation: 493

For CPU percentage

avg((sum (rate (container_cpu_usage_seconds_total {container_name!="" ,pod="<Pod name>" } [5m])) by (namespace , pod, container ) / on (container , pod , namespace) ((kube_pod_container_resource_limits_cpu_cores >0)*300))*100)

For Memory percentage

avg((avg (container_memory_working_set_bytes{pod="<pod name>"}) by (container_name , pod ))/ on (container_name , pod)(avg (container_spec_memory_limit_bytes>0 ) by (container_name, pod))*100)

you can use above promql with pod name in a query.

Upvotes: 15

Максим Штернберг
Максим Штернберг

Reputation: 84

Do you use prometheus-operator to collect data from kubernetes? If yes, you can use something like this: sum(container_memory_usage_bytes) sum(container_cpu_usage_seconds_total) Just for example.

Upvotes: 0

Related Questions