CODEWITHSUNDEEP
kubernetes
DenCowboy
DenCowboy

Reputation: 15066

Kubernetes: how to scale my pods

I'm new to Kubernetes. I try to scale my pods. First I started 3 pods:

./cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80

There were starting 3 pods. First I tried to scale up/down by using a replicationcontroller but this did not exist. It seems to be a replicaSet now.

./cluster/kubectl.sh get rs
NAME                  DESIRED   CURRENT   AGE
my-nginx-2494149703   3         3         9h

I tried to change the amount of replicas described in my replicaset:

./cluster/kubectl.sh scale --replicas=5 rs/my-nginx-2494149703
replicaset "my-nginx-2494149703" scaled

But I still see my 3 original pods

./cluster/kubectl.sh get pods
NAME                        READY     STATUS    RESTARTS   AGE
my-nginx-2494149703-04xrd   1/1       Running   0          9h
my-nginx-2494149703-h3krk   1/1       Running   0          9h
my-nginx-2494149703-hnayu   1/1       Running   0          9h

I would expect to see 5 pods. 

./cluster/kubectl.sh describe rs/my-nginx-2494149703
Name:       my-nginx-2494149703
Namespace:  default
Image(s):   nginx
Selector:   pod-template-hash=2494149703,run=my-nginx
Labels:     pod-template-hash=2494149703
        run=my-nginx
Replicas:   3 current / 3 desired
Pods Status:    3 Running / 0 Waiting / 0 Succeeded / 0 Failed

Why isn't it scaling up? Do I also have to change something in the deployment?

I see something like this when I describe my rs after scaling up: (Here I try to scale from one running pod to 3 running pods). But it remains one running pod. The other 2 are started and killed immediatly

  34s       34s     1   {replicaset-controller }            Normal      SuccessfulCreate    Created pod: my-nginx-1908062973-lylsz
  34s       34s     1   {replicaset-controller }            Normal      SuccessfulCreate    Created pod: my-nginx-1908062973-5rv8u
  34s       34s     1   {replicaset-controller }            Normal      SuccessfulDelete    Deleted pod: my-nginx-1908062973-lylsz
  34s       34s     1   {replicaset-controller }            Normal      SuccessfulDelete    Deleted pod: my-nginx-1908062973-5rv8u

Upvotes: 44

Views: 157665

Answers (8)

Rotem jackoby
Rotem jackoby

Reputation: 22058

Autoscaling

All answers are referring to manual scaling and interaction with kubectl.
It is also worth to mention the option of HPA (Horizontal Pod Autoscaling).

In general

In Kubernetes, a HorizontalPodAutoscaler automatically updates a workload resource (such as a Deployment or StatefulSet), with the aim of automatically scaling the workload to match demand.

Horizontal scaling means that the response to increased load is to deploy more Pods. This is different from vertical scaling, which for Kubernetes would mean assigning more resources (for example: memory or CPU) to the Pods that are already running for the workload.

If the load decreases, and the number of Pods is above the configured minimum, the HorizontalPodAutoscaler instructs the workload resource (the Deployment, StatefulSet, or other similar resource) to scale back down.

Horizontal pod autoscaling does not apply to objects that can't be scaled (for example: a DaemonSet.)

The HorizontalPodAutoscaler is implemented as a Kubernetes API resource and a controller. The resource determines the behavior of the controller. The horizontal pod autoscaling controller, running within the Kubernetes control plane, periodically adjusts the desired scale of its target (for example, a Deployment) to match observed metrics such as average CPU utilization, average memory utilization, or any other custom metric you specify.

Common usage

The common use for HorizontalPodAutoscaler is to configure it to fetch metrics from aggregated APIs (metrics.k8s.io, custom.metrics.k8s.io, or external.metrics.k8s.io).

The metrics.k8s.io API is usually provided by an add-on named Metrics Server, which needs to be launched separately.
For more information about resource metrics, see Metrics Server.

Example

An example for scaling on CPU and Memory policy:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 100Mi

Read about the Configurable scaling behavior.

Additional information:

  1. Ensure you're using apiVersion: autoscaling/v2.
  2. See here that you can also scale on Custom Metrics.
  3. In case you need to scale from/to 0 or scale according to some Cloud resources metrics (like AWS SQS, GCP pub/Sub or other services), Kafka, Redis and other - read about Keda.

Upvotes: 0

Milad Jahandideh
Milad Jahandideh

Reputation: 751

for deployment

kubectl scale deployment <deployment-name> --replicas=3 -n <namespace>

for statefulset

kubectl scale statefulsets <stateful-set-name> --replicas=3 -n <namespace>

Upvotes: 5

Harsimranjit Singh Kler
Harsimranjit Singh Kler

Reputation: 2302

This is working for me

kubectl scale --replicas=<expected_replica_num> deployment <deployment_label_name> -n <namespace>

Example

# kubectl scale --replicas=3 deployment xyz -n my_namespace

Upvotes: 71

AATHITH RAJENDRAN
AATHITH RAJENDRAN

Reputation: 5366

kubectl run my-nginx --image=nginx --replicas=3 --port=80 in this kubectl run will create a deployment or job to manage the created container(s).
Deployment-->ReplicaSet-->Pod this is how kubernetes works.
If you change the bottom-level object, its higher-level object will undo your change.You have to change the top-level object.

enter image description here

Upvotes: 27

Isuru Amarathunga
Isuru Amarathunga

Reputation: 2397

scale it down to zero and then to the number of pods you required (guess it equals to 3)

kubectl scale deployment <deployment-name> --replicas=0 -n <namespace>
kubectl scale deployment <deployment-name> --replicas=3 -n <namespace>

Upvotes: 11

Deb
Deb

Reputation: 663

Below example shows how you should scale up/down your "pods/resource/deployments".

k8smaster@k8smaster:~/debashish$ more createdeb_deployment1.yaml 


--- 
apiVersion: apps/v1beta2
kind: Deployment
metadata: 
  name: debdeploy-webserver
spec: 
  replicas: 1
  selector: 
    matchLabels: 
      app: debdeploy1webserver
  template: 
    metadata: 
      labels: 
        app: debdeploy1webserver
    spec: 
      containers: 
        - 
          image: "docker.io/debu3645/debapachewebserver:v1"
          name: deb-deploy1-container 
          ports: 
            - 
              containerPort: 6060

deployment created --> 

**kubectl -n debns1 create -f createdeb_deployment1.yaml**




k8smaster@k8smaster:~/debashish$ `kubectl scale --replicas=5 **deployment**/debdeploy-webserver -n debns1`

(Scale up 5 deployments)

k8smaster@k8smaster:~/debashish$ kubectl get pods -n debns1


NAME                                   READY   STATUS    RESTARTS   AGE
debdeploy-webserver-7cf4fb74c5-8wvzx   1/1     Running   0          16s
debdeploy-webserver-7cf4fb74c5-jrf6v   1/1     Running   0          16s
debdeploy-webserver-7cf4fb74c5-m9fpw   1/1     Running   0          16s
debdeploy-webserver-7cf4fb74c5-q9n7r   1/1     Running   0          16s
debdeploy-webserver-7cf4fb74c5-ttw6p   1/1     Running   1          19h
resourcepod-deb1                       1/1     Running   5          6d18h




k8smaster@k8smaster:~/debashish$ **kubectl get ep -n debns1**



NAME                ENDPOINTS                                                     AGE
frontend-svc-deb    192.168.1.10:80,192.168.1.11:80,192.168.1.12:80 + 2 more...   18h
frontend-svc1-deb   192.168.1.8:80                                                14d
frontend-svc2-deb   192.168.1.8:80                                                5d19h




k8smaster@k8smaster:~/debashish$ **kubectl scale --replicas=2** deployment/debdeploy-webserver -n debns1

(Scale down from 5 to 2)

deployment.extensions/debdeploy-webserver scaled

k8smaster@k8smaster:~/debashish$ **kubectl get pods -n debns1**


NAME                                   READY   STATUS        RESTARTS   AGE
debdeploy-webserver-7cf4fb74c5-8wvzx   1/1     Terminating   0          35m
debdeploy-webserver-7cf4fb74c5-jrf6v   1/1     Terminating   0          35m
debdeploy-webserver-7cf4fb74c5-m9fpw   1/1     Terminating   0          35m
debdeploy-webserver-7cf4fb74c5-q9n7r   1/1     Running       0          35m
debdeploy-webserver-7cf4fb74c5-ttw6p   1/1     Running       1          19h
resourcepod-deb1                       1/1     Running       5          6d19h


k8smaster@k8smaster:~/debashish$ **kubectl get pods -n debns1**


NAME                                   READY   STATUS    RESTARTS   AGE
debdeploy-webserver-7cf4fb74c5-q9n7r   1/1     Running   0          37m
debdeploy-webserver-7cf4fb74c5-ttw6p   1/1     Running   1          19h
resourcepod-deb1                       1/1     Running   5          6d19h

k8smaster@k8smaster:~/debashish$ kubectl **scale --current-replicas=4 --replicas=2** deployment/debdeploy-webserver -n debns1  (Check the current no. of deployments. If current replication is 4, then bring it down to 2, else dont do anything)


error: Expected replicas to be 4, was 2


k8smaster@k8smaster:~/debashish$ **kubectl scale --current-replicas=3 --replicas=10 deployment/debdeploy-webserver -n debns1**


error: Expected replicas to be 3, was 2


k8smaster@k8smaster:~/debashish$ **kubectl scale --current-replicas=2 --replicas=10 deployment/debdeploy-webserver -n debns1**

deployment.extensions/debdeploy-webserver scaled

k8smaster@k8smaster:~/debashish$ **kubectl get pods -n debns1**


    NAME                                   READY   STATUS              RESTARTS   AGE
    debdeploy-webserver-7cf4fb74c5-46bxg   1/1     Running             0          6s
    debdeploy-webserver-7cf4fb74c5-d6qsx   0/1     ContainerCreating   0          6s
    debdeploy-webserver-7cf4fb74c5-fdq6v   1/1     Running             0          6s
    debdeploy-webserver-7cf4fb74c5-gd87t   1/1     Running             0          6s
    debdeploy-webserver-7cf4fb74c5-kqdbj   0/1     ContainerCreating   0          6s
    debdeploy-webserver-7cf4fb74c5-q9n7r   1/1     Running             0          47m
    debdeploy-webserver-7cf4fb74c5-qjvm6   1/1     Running             0          6s
    debdeploy-webserver-7cf4fb74c5-skxq4   0/1     ContainerCreating   0          6s
    debdeploy-webserver-7cf4fb74c5-ttw6p   1/1     Running             1          19h
    debdeploy-webserver-7cf4fb74c5-wlc7q   0/1     ContainerCreating   0          6s
    resourcepod-deb1                       1/1     Running             5          6d19h

Upvotes: 1

Harry Moreno
Harry Moreno

Reputation: 11593

Not sure if this is the best way as I'm starting out with kubernetes, but I did this by updating my yaml file

# app.yaml
apiVersion: apps/v1
...
spec:
  replicas: <new value>

and running $ kubectl scale -f app.yaml --replicas=<new value>

you can verify your new number of replicas by running $ kubectl get pods

In my case I was also interested in scaling back my VMs, on google cloud. I did this with $ gcloud container clusters resize appName --size=1 --zone "my-zone"

Upvotes: 2

Robert Bailey
Robert Bailey

Reputation: 18200

TL;DR: You need to scale your deployment instead of the replica set directly.

If you try to scale the replica set, then it will (for a very short time) have a new count of 5. But the deployment controller will see that the current count of the replica set is 5 and since it knows that it is supposed to be 3, it will reset it back to 3. By manually modifying the replica set that was created for you, you are fighting with the system controller (which is untiring and will pretty much always outlast you).

Upvotes: 45

Related Questions