CODEWITHSUNDEEP
kuberneteskubernetes-statefulsetkubernetes-pvc
yborgess
yborgess

Reputation: 334

Is there any technical problem if I have one PVC to share the same volume across all statefulset replicas?

Kubernetes creates one PersistentVolume for each VolumeClaimTemplate definition on an statefulset. That makes each statefulset pod have its own storage that is not shared across the replicas. However, I would like to share the same volume across all the statefulset replicas.

It looks like the approach should be the following:

  1. Create a PVC on the same namespace.
  2. On the statefulset use Volumes to bound the PVC
  3. Ensure that the PVC is ReadOnlyMany or ReadWriteMany

Assuming that my application is able to deal with any concurrency on the shared volume, is there any technical problem if I have one PVC to share the same volume across all statefulset replicas?

Upvotes: 1

Views: 1477

Answers (1)

Dawid Kruk
Dawid Kruk

Reputation: 9885

I wholeheartedly agree with the comments made by @Jonas and @David Maze:

You can do this, it should work. There is no need to use volumeClaimTemplates unless your app needs it.

Two obvious problems are that ReadWriteMany volumes are actually a little tricky to get (things like AWS EBS volumes are only ReadWriteOnce), and that many things you want to run in StatefulSets (like databases) want exclusive use of their filesystem space and use file locking to enforce this.

Answering on the question:

Is there any technical problem if I have one PVC to share the same volume across all statefulset replicas?

I'd say that this would mostly depend on the:

  • How the application would handle such scenario where it's having single PVC (writing concurrency).
  • Which storage solution are supported by your Kubernetes cluster (or could be implemented).

Subjectively speaking, I don't think there should be an issue when above points are acknowledged and aligned with the requirements and configuration that the cluster/applications allows.

From the application perspective, there is an inherent lack of the software we are talking about. Each application could behave differently and could require different tuning (look on the David Maze comment).

We do not also know anything about your infrastructure so it could be hard to point you potential issues. From the hardware perspective (Kubernetes cluster), this would inherently go into making a research on the particular storage solution that you would like to use. It could be different from cloud provider to cloud provider as well as on-premise solutions. You would need to check the requirements of your app to align it to the options you have.

Continuing on the matter of Volumes, I'd reckon the one of the important things would be accessModes.

Citing the official docs:

Access Modes

A PersistentVolume can be mounted on a host in any way supported by the resource provider. As shown in the table below, providers will have different capabilities and each PV's access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV's capabilities.

The access modes are:

  • ReadWriteOnce -- the volume can be mounted as read-write by a single node
  • ReadOnlyMany -- the volume can be mounted read-only by many nodes
  • ReadWriteMany -- the volume can be mounted as read-write by many nodes

In the CLI, the access modes are abbreviated to:

  • RWO - ReadWriteOnce
  • ROX - ReadOnlyMany
  • RWX - ReadWriteMany

One of the issues you can run into is with the ReadWriteOnce when the PVC is mounted to the Node and sts-X (Pod) is scheduled onto a different Node but from the question, I'd reckon you already know about it.

However, I would like to share the same volume across all the statefulset replicas.

An example of a StatefulSet with a Volume that would be shared across all of the replicas could be following (modified example from Kubernetes documentation):

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx # has to match .spec.template.metadata.labels
  serviceName: "nginx"
  replicas: 3 # by default is 1
  template:
    metadata:
      labels:
        app: nginx # has to match .spec.selector.matchLabels
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web
      # VOLUME START 
        volumeMounts:
        - name: example-pvc
          mountPath: /usr/share/nginx/html
      volumes:
       - name: example-pvc
         persistentVolumeClaim:
           claimName: pvc-for-sts
      # VOLUME END

Additional resources:

Upvotes: 2

Related Questions