CODEWITHSUNDEEP
kubernetes
stackoverflowed
stackoverflowed

Reputation: 927

Kubernetes PV refuses to bind after delete/re-create

Given the following PVC and PV:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: packages-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeName: packages-volume

apiVersion: v1
kind: PersistentVolume
metadata:
  name: packages-volume
  namespace: test
spec:
  claimRef:
    name: packages-pvc
    namespace: test
  accessModes:
    - ReadWriteMany
  nfs:
    path: {{NFS_PATH}}
    server: {{NFS_SERVER}}
  capacity:
    storage: 1Gi
  persistentVolumeReclaimPolicy: Retain

if I create the PV, then the PVC, they bind together. However if I delete the PVC then re-create it, they do not bind (pvc pending). Why?

Upvotes: 5

Views: 3677

Answers (2)

Sandeep kumar singh
Sandeep kumar singh

Reputation: 685

the above answer solves the problem. however there is a reason that PV is kept into released stated. k8s controller can easily remove the PVC ref after the user deletes the PVC claim and brings the PV to be in available state.

But there is a problem, once a PV goes into bound state, it's must have some data. if you forcefully bind it, you will loose or corrupt your data.

so you must backup/clean the PV, before you try to make it available.

thanks

Upvotes: 0

mario
mario

Reputation: 11138

Note that after deleting PVC, PV remains in Released status:

$ kubectl get pv packages-volume
NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                  STORAGECLASS   REASON   AGE
packages-volume   1007Gi     RWX            Retain           Released   default/packages-pvc                           10m

It should have status Available so it can be reused by another PersistentVolumeClaim instance.

Why it isn't Available ?

If you display current yaml definition of the PV, which you can easily do by executing:

kubectl get pv packages-volume -o yaml

you may notice that in claimRef section it contains the uid of the recently deleted PersistentVolumeClaim:

  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: packages-pvc
    namespace: default
    resourceVersion: "10218121"
    uid: 1aede3e6-eaa1-11e9-a594-42010a9c0005

You can easily verify it by issuing:

kubectl get pvc packages-pvc -o yaml | grep uid

just before you delete your PVC and compare it with what PV definition contains. You'll see that this is exactly the same uid that is still referred by your PV after PVC is deleted. This remaining reference is the actual reason that PV remains in Released status.

Why newly created PVC remains in a Pending state ?

Although your newly created PVC may seem to you exactly the same PVC that you've just deleted as you're creating it using the very same yaml file, from the perspective of Kubernetes it's a completely new instance of PersistentVolumeClaim object which has completely different uid. This is the reason why it remains in Pending status and is unable to bind to the PV.

Solution:

To make the PV Available again you just need to remove the mentioned uid reference e.g. by issuing:

kubectl patch pv packages-volume --type json -p '[{"op": "remove", "path": "/spec/claimRef/uid"}]'

or alternatively by removing the whole claimRef section which can be done as follows:

kubectl patch pv packages-volume --type json -p '[{"op": "remove", "path": "/spec/claimRef"}]'

Upvotes: 28

Related Questions