CODEWITHSUNDEEP
kubernetes
karthikeayan
karthikeayan

Reputation: 5000

How to find available resources in a Kubernetes Cluster level?

Do we have a simple command through which we can find the available resources at cluster level?

Available CPU and Memory requests.

Upvotes: 14

Views: 42123

Answers (4)

nichoio
nichoio

Reputation: 7587

You have to be careful about the distinction between "allocatable" and "unallocated". Every node has certain amounts of resources for memory, CPU and ephemeral storage which are allocatable. From the Kubernetes docs on allocatable resources:

'Allocatable' on a Kubernetes node is defined as the amount of compute resources that are available for pods. The scheduler does not over-subscribe 'Allocatable'.

And that's it. The field .status.allocatable of a node refers to resources that are available generally, ignoring resources requested by pods which are scheduled onto that node right now.

For a vanilla Kubernetes cluster without any metrics-server running (which is an optional component), if we want to know what resources on a given node are unallocated right now, we can use the data provided by "Allocated" field by running kubectl describe node-name. Example output:

(...)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests   Limits
  --------           --------   ------
  cpu                75m (7%)   0 (0%)
  memory             50Mi (2%)  170Mi (9%)
  ephemeral-storage  0 (0%)     0 (0%)
  hugepages-2Mi      0 (0%)     0 (0%)
(...)

To get the amount of unallocated resource, you do have to do the math of yourself, by subtracting allocated resources from allocatable resources.

Upvotes: 4

taitelman
taitelman

Reputation: 700

why don't you try https://github.com/robscott/kube-capacity ? it has a nice CLI for what you need. pay special attention to the first row (star) this is what you asked for.

$ kube-capacity
NODE            CPU REQUESTS    CPU LIMITS        MEMORY REQUESTS   MEMORY LIMITS
*               236888m (47%)   2595014m (518%)   457068Mi (53%)    1225718Mi (142%)
10.184.249.15   1690m (5%)      7400m (23%)       5691Mi (9%)       8218Mi (14%)
10.184.249.16   2805m (71%)     11400m (291%)     6873Mi (52%)      12314Mi (93%)
10.184.249.17   3827m (24%)     12626m (79%)      9140Mi (69%)      12326Mi (93%)
10.184.249.22   5093m (32%)     16403m (103%)     9403Mi (71%)      23586Mi (179%)
10.184.249.27   3805m (97%)     13400m (342%)     7897Mi (60%)      15386Mi (117%)
10.184.249.58   21567m (67%)    271848m (853%)    38922Mi (66%)     113074Mi (193%)
10.184.249.6    20038m (62%)    263452m (827%)    38246Mi (65%)     115507Mi (197%)
10.184.249.8    22123m (69%)    274492m (862%)    38547Mi (65%)     115766Mi (197%)
10.73.80.203    1690m (5%)      7400m (23%)       5691Mi (9%)       8218Mi (14%)
10.73.80.209    3700m (94%)     11400m (291%)     7759Mi (59%)      14362Mi (109%)
10.73.80.218    21984m (69%)    273956m (860%)    39035Mi (66%)     114884Mi (196%)
10.73.80.220    22262m (69%)    254028m (797%)    34644Mi (59%)     107426Mi (183%)
10.73.80.223    19482m (61%)    281808m (885%)    42036Mi (71%)     122037Mi (208%)
10.73.80.238    3293m (20%)     10903m (68%)      6991Mi (53%)      11746Mi (89%)
10.73.80.239    1800m (46%)     9400m (240%)      5839Mi (44%)      9242Mi (70%)
10.73.80.243    3698m (23%)     12603m (79%)      8715Mi (66%)      15306Mi (116%)
10.93.203.131   1690m (5%)      7400m (23%)       5691Mi (9%)       8218Mi (14%)
10.93.203.133   2713m (17%)     13103m (82%)      6693Mi (50%)      11922Mi (90%)
10.93.203.160   3700m (94%)     11400m (291%)     7759Mi (59%)      14362Mi (109%)
10.93.203.162   3700m (94%)     11400m (291%)     7759Mi (59%)      14362Mi (109%)
10.93.203.180   21011m (65%)    278804m (875%)    39823Mi (67%)     119195Mi (203%)
10.93.203.183   2500m (15%)     9400m (59%)       8032Mi (61%)      13114Mi (99%)
10.93.203.184   21289m (66%)    279276m (877%)    39767Mi (67%)     119025Mi (203%)
10.93.203.188   21428m (67%)    251712m (790%)    36126Mi (61%)     106127Mi (181%)

Upvotes: 1

PjoterS
PjoterS

Reputation: 14084

There are couple of ways to achieve this. You didn't mention what environment are you using, however you probably already have metric server in your cluster.

1. Top command

kubectl top pods or kubectl top nodes. This way you will be able to check current usage of pods/nodes. You can also narrow it to namespace.

2. Describe node

If you will execute kubectl describe node, in output you will be able to see Capacity of that node and how much allocated resources left. Similar with Pods.

...
Capacity:
 attachable-volumes-gce-pd:  127
 cpu:                        1
 ephemeral-storage:          98868448Ki
 hugepages-2Mi:              0
 memory:                     3786684Ki
 pods:                       110
Allocatable:
 attachable-volumes-gce-pd:  127
 cpu:                        940m
 ephemeral-storage:          47093746742
 hugepages-2Mi:              0
 memory:                     2701244Ki
 pods:                       110
 ...

3. Prometheus

If you need more detailed information with statistics, I would recommend you to use Prometheus. It will allow you to create statistics of nodes/pods, generate alerts and many more. It also might provide metrics not only CPU and Memory but also custom.metrics which can create statistics of all Kubernetes objects.

Many useful information can be found here.

Upvotes: 15

shubham_asati
shubham_asati

Reputation: 687

this is very useful script to monitor kubernetes resources 

https://www.jeffgeerling.com/blog/2019/monitoring-kubernetes-cluster-utilization-and-capacity-poor-mans-way

i would suggest to use prometheus to get such metrics

Upvotes: 7

Related Questions