How to expose a headless service for a StatefulSet cassandra cluster externally in Kubernetes

Question

I'm setting up multi node cassandra cluster in kubernetes (Azure AKS),Since this a headless service with statefull set pods without having a external IP. How can i connect my spark application with cassandra which is in kubernetes cluster

We have tried with cluster ip,ingress ip also but only single pod is getting up rest are failing.

I have 3 manifest:

1. Service
2. PersistentVolumeClaim
3. StatefulSet

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  clusterIP: None
  ports:
    - port: 9042
  selector:
    app: cassandra


kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myvolume-disk-claim
spec:
  storageClassName: default
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi


apiVersion: "apps/v1"
kind: StatefulSet
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  serviceName: cassandra
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
        - name: cassandra
          image: gcr.io/google-samples/cassandra:v13
          imagePullPolicy: Always
          ports:
            - containerPort: 7000
              name: intra-node
            - containerPort: 7001
              name: tls-intra-node
            - containerPort: 7199
              name: jmx
            - containerPort: 9042
              name: cql
          env:
            - name: CASSANDRA_SEEDS
              value: cassandra-0.cassandra.default.svc.cluster.local
            - name: MAX_HEAP_SIZE
              value: 256M
            - name: HEAP_NEWSIZE
              value: 100M
            - name: CASSANDRA_CLUSTER_NAME
              value: "Cassandra"
            - name: CASSANDRA_DC
              value: "DC1"
            - name: CASSANDRA_RACK
              value: "Rack1"
            - name: CASSANDRA_ENDPOINT_SNITCH
              value: GossipingPropertyFileSnitch
            - name: POD_IP
              valueFrom:
                fieldRef:
                   fieldPath: status.podIP
          readinessProbe:
            exec:
              command:
              - /bin/bash
              - -c
              - /ready-probe.sh
            initialDelaySeconds: 15
            timeoutSeconds: 5        
          volumeMounts:
          - mountPath: /var/lib/cassandra/data
            name: myvolume-disk-claim
  volumeClaimTemplates:
  - metadata:
      name: myvolume-disk-claim
    spec:
      storageClassName: default
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

Expected Result:(public ip as external IP)

dspg@Digiteds28:$ kubectl get svc
NAME                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes                ClusterIP   10.0.0.1     <none>        443/TCP    1h
cassandra                 ClusterIP   None        154.167.90.98  9042/TCP   1h



dspg@Digiteds28:$ kubectl get pod
NAME                READY   STATUS    RESTARTS   AGE
cassandra-0         1/1     Running   0          59m
cassandra-1         1/1     Running   0          58m
cassandra-2         1/1     Running   0          56m

Actual Output:

dspg@Digiteds28:$ kubectl get svc
NAME                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
kubernetes                ClusterIP   10.0.0.1     <none>        443/TCP    1h
cassandra                 ClusterIP   None         <none>        9042/TCP   1h



dspg@Digiteds28:$ kubectl get pod
NAME                READY   STATUS    RESTARTS   AGE
cassandra-0         1/1     Running   0          59m
cassandra-1         1/1     Running   0          58m
cassandra-2         1/1     Running   0          56m

Now this doesnot include external IP to connect to application.

Answer 1

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra-ext
spec:
  type: LoadBalancer
  ports:
    - port: 9042
  selector:
    app: cassandra

Using the additonal service gave me a External IP and because if which i started connecting my spark or devcenter application to cassandra cluster.

Answer 2

It depends on what exactly you are trying to do. If you need an external IP then in general you'd need to create an additional Service object (probably type: LoadBalancer) like this:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra-ext
spec:
  type: LoadBalancer
  ports:
    - port: 9042
  selector:
    app: cassandra

If you need to reach it from within the cluster then use the DNS name cassandra-0.cassandra.default from the other pod (if the StatefulSet was deployed in the default namespace)