elasticsearch cluster members are not turning into ready state in kubernetes

Question

i am a begginer in kubernetes i am trying to run an elasticsearch cluster in it for nthat i am following this (tutorial)[https://phoenixnap.com/kb/elasticsearch-helm-chart] followed all the commands as it is and in the ebnd when i do

kubectl get pods --namespace=default -l app=elasticsearch-master -w

i am getting

NAME                     READY   STATUS     RESTARTS   AGE
elasticsearch-master-0   0/1     Init:0/1   0          55s
elasticsearch-master-1   0/1     Init:0/1   0          55s
elasticsearch-master-2   0/1     Init:0/1   0          55s

they are not turning into ready state and the command

helm test elasticsearch

got stuck and does not respond

here are the logs

kubectl logs -f elasticsearch-master-0
Error from server (BadRequest): container "elasticsearch" in pod "elasticsearch-master-0" is waiting to start: PodInitializing
 kubectl logs -f elasticsearch-master-1
Error from server (BadRequest): container "elasticsearch" in pod "elasticsearch-master-1" is waiting to start: PodInitializing
 kubectl logs -f elasticsearch-master-2
Error from server (BadRequest): container "elasticsearch" in pod "elasticsearch-master-2" is waiting to start: PodInitializing

and the kubectl cluster-info returning

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "services \"kube-dns:dns\" is forbidden: User \"system:anonymous\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"kube-system\"",
  "reason": "Forbidden",
  "details": {
    "name": "kube-dns:dns",
    "kind": "services"
  },
  "code": 403
}

that's all i know right now as i am a begenier in it please help why my elasticsearch nodes are not getting ready

Answer 1

There is an open issue around this helm on GitHub.

Check once which storage class you have for PVC creation. 

however, still, there could be multiple reasons for this issue like PVC is not getting created or others.

You can try if single node Elasticsearch working or not with PVC dynamic creation using :

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app : elasticsearch
    component: elasticsearch
    release: elasticsearch
  name: elasticsearch
spec:
  podManagementPolicy: Parallel
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app : elasticsearch
      component: elasticsearch
      release: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      creationTimestamp: null
      labels:
        app : elasticsearch
        component: elasticsearch
        release: elasticsearch
    spec:
      containers:
      - env:
        - name: cluster.name
          value: <SET THIS>
        - name: discovery.type
          value: single-node
        - name: ES_JAVA_OPTS
          value: -Xms512m -Xmx512m
        - name: bootstrap.memory_lock
          value: "false"
        image: elasticsearch:6.5.0
        imagePullPolicy: IfNotPresent
        name: elasticsearch
        ports:
        - containerPort: 9200
          name: http
          protocol: TCP
        - containerPort: 9300
          name: transport
          protocol: TCP
        resources:
          limits:
            cpu: 250m
            memory: 1Gi
          requests:
            cpu: 150m
            memory: 512Mi
        securityContext:
          privileged: true
          runAsUser: 1000
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      dnsPolicy: ClusterFirst
      initContainers:
      - command:
        - sh
        - -c
        - chown -R 1000:1000 /usr/share/elasticsearch/data
        - sysctl -w vm.max_map_count=262144
        - chmod 777 /usr/share/elasticsearch/data
        - chomod 777 /usr/share/elasticsearch/data/node
        - chmod g+rwx /usr/share/elasticsearch/data
        - chgrp 1000 /usr/share/elasticsearch/data
        image: busybox:1.29.2
        imagePullPolicy: IfNotPresent
        name: set-dir-owner
        resources: {}
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 10
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      creationTimestamp: null
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi