CODEWITHSUNDEEP
kubernetespersistent-volumespersistent-storagekubernetes-pvcpersistent-volume-claims
Andrei Manolache
Andrei Manolache

Reputation: 933

Kubernetes pod cannot attach to in-use volume

I have a deployment (see minimal config below) with 2 replicas which write their logs to a volume provisioned via PVC. My problem is that if I try to upscale the number of replicas / update the deployment, then I receive an error that the pods cannot be attached to the volume:

0s          Warning   FailedAttachVolume       pod/POD_NAME
AttachVolume.Attach failed for volume "pvc-XXXXXX" : rpc error: code =
Internal desc = [ControllerPublishVolume] Attach Volume failed with error
failed to attach XXXXXX volume to XXXXXXXX compute: Bad request with: [POST
https://SOME_URL], error message: {"badRequest": {"code": 400, "message":
"Invalid input received: Invalid volume: Volume XXXXXXXX status must be
available or downloading to reserve, but the current status is in-use.
(HTTP 400) (Request-ID: XXXXXXX)"}}

PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  storageClassName: csi-cinder-tiefighter

deployment (some metadata was omitted because it's useless):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: my-label
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: my-label
      shareProcessNamespace: true
      initContainers:
        - name: add-volume-log-permissions
          image: my-busybox-iamge
          command: [ "/bin/sh" ]
          args: [ "-c", "chown 1000:1000 MY_PATH" ]
          securityContext:
            runAsUser: 0
          volumeMounts:
            - mountPath: MY_PATH
              name: volume-logs
          resources:
            limits:
              memory: 100Mi
              cpu: 100m
      containers:
      - name: my-container
        image: MY_IMAGE
        imagePullPolicy: Always
        volumeMounts:
          - mountPath: MY_PATH
            name: volume-logs
      volumes:
        - name: volume-logs
          persistentVolumeClaim:
            claimName: myPvc

Now, how do I overcome this? I need the volume to store my logs and the deployment should have minimum 2 replicas.

Upvotes: 0

Views: 777

Answers (2)

x-zone-cat
x-zone-cat

Reputation: 552

Setup your deployment app with kind: Statefulset and use Volume Claim Templates for your persistent volumes to be provisioned by PersistentVolume Provisioner for dynamic PVC provisioning.

I also noticed that you used accessModes: ReadWriteMany wherein you want a PVC on your pods, ReadWriteOnce is more applicable unless you do have NFS for storage sharing.

Upvotes: 0

Othniel Karlax
Othniel Karlax

Reputation: 11

Change the access mode of your PVC from Readwritemany to Readwriteonce

After that you will try to increase, it might work

Upvotes: 1

Related Questions