CODEWITHSUNDEEP
kubernetesansiblekubectl
Z.Liu
Z.Liu

Reputation: 423

ansible verify the k8s deployment succeed

I am using ansible k8s module to do the pod deployment, but unfortunately, I did not find a way to use wait and wait_condition correctly.

so I tried use the until and retry to implement another waiting

each time after you run the playbook, after you use k8s module to do the deployment, until NewReplicaSetAvailable shows up in the deployment within 2 minutes(60 * 2s), otherwise the deployment failed.

In case you need to verify the deployment result, I write down here.

    - name: Verify the deploy result
      shell: "kubectl -n kube-system get deployment coredns --output=jsonpath='{.status.conditions[*].reason}'"
      register: deploy_res
      until: "'NewReplicaSetAvailable' in deploy_res.stdout"
      retries: 60
      delay: 2

Upvotes: 3

Views: 3587

Answers (2)

ankuj
ankuj

Reputation: 21

Validation can be done like number of replica set are equal or not, imgae updated on all pods.

    - name: Add a deployment
      k8s:
        definition:
          apiVersion: apps/v1
          kind: Deployment
          metadata:
            name: wait-deploy
            namespace: "{{ wait_namespace }}"
          spec:
            replicas: 3
            selector:
              matchLabels:
                app: "{{ k8s_pod_name }}"
            template: "{{ k8s_pod_template }}"
        wait: yes
        wait_timeout: "{{ k8s_wait_timeout | default(omit) }}"
      vars:
        k8s_pod_name: wait-deploy
        k8s_pod_image: gcr.io/kuar-demo/kuard-amd64:1
        k8s_pod_ports:
          - containerPort: 8080
            name: http
            protocol: TCP

      register: deploy

    - name: Check that deployment wait worked
      assert:
        that:
          - deploy.result.status.availableReplicas == deploy.result.status.replicas

    - name: Update a deployment
      k8s:
        definition:
          apiVersion: apps/v1
          kind: Deployment
          metadata:
            name: wait-deploy
            namespace: "{{ wait_namespace }}"
          spec:
            replicas: 3
            selector:
              matchLabels:
                app: "{{ k8s_pod_name }}"
            template: "{{ k8s_pod_template }}"
        wait: yes
        wait_timeout: "{{ k8s_wait_timeout | default(omit) }}"
      vars:
        k8s_pod_name: wait-deploy
        k8s_pod_image: gcr.io/kuar-demo/kuard-amd64:2
        k8s_pod_ports:
          - containerPort: 8080
            name: http
            protocol: TCP
      register: update_deploy

    # It looks like the Deployment is updated to have the desired state *before* the pods are terminated
    # Wait a couple of seconds to allow the old pods to at least get to Terminating state
    - name: Avoid race condition
      pause:
        seconds: 2

    - name: Get updated pods
      k8s_info:
        api_version: v1
        kind: Pod
        namespace: "{{ wait_namespace }}"
        label_selectors:
          - app=wait-deploy
        field_selectors:
          - status.phase=Running
      register: updated_deploy_pods
      until: updated_deploy_pods.resources[0].spec.containers[0].image.endswith(':2')
      retries: 6
      delay: 5

    - name: Check that deployment wait worked
      assert:
        that:
          - deploy.result.status.availableReplicas == deploy.result.status.replicas

Mention solution can be found on kubernetes Ansible repo k8s_waiter

Upvotes: 2

CLNRMN
CLNRMN

Reputation: 1457

the module k8s gives you not the status of the Deployment. For the status of a Deployment (or any other resource) you should use the module k8s_info. Then you can do your check on this task.

Example:

---
- hosts: localhost
  tasks:
  - name: Create a Service object from an inline definition
    k8s:
      state: present
      definition:
        apiVersion: v1
        kind: Deployment
        metadata:
          name: example
          namespace: default
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: example-nginx
          template:
            metadata:
              labels:
                app: example-nginx
            spec:
              containers:
              - image: nginx
                name: nginx
  - name: check if deployment is ready
    k8s_info:
      kind: Deployment
      label_selectors: 
        - app = example-nginx
    register: output_info
    until: output_info.resources | json_query('[*].status.conditions[?reason==`NewReplicaSetAvailable`][].status') | select ('match','True') | list | length == 1
    delay: 2
    retries: 5

For the until filter there are certainly better solutions, but I have solved it this way for a similar use case. The until filter checks if there is a condition with the reason NewReplicaSetAvailable and it must be True. If that happens, the length is equal to 1

Upvotes: 4

Related Questions