CODEWITHSUNDEEP
kubernetesvmwarerancherpersistent-volumes
Franklin Piat
Franklin Piat

Reputation: 4561

Rancher Kubernetes can't create persisten volume claim

I can't create Persistent volume claim in Kubernetes (Rancher 2.3).

My storage class use VMware cloud provider 'vSphere Storage for Kubernetes' provided by Rancher

In rancher web interface, the Event show errors like:

(combined from similar events): Failed to provision volume with StorageClass "t-a-g1000-t6m-e0fd": Post https://vsphere.exemple.com:443/sdk: dial tcp: lookup vsphere.exemple.com on [::1]:53: read udp [::1]:51136->[::1]:53: read: connection refused

I get the same error on my Kubernetes Master:

docker logs kube-controller-manager

Upvotes: 0

Views: 429

Answers (1)

Franklin Piat
Franklin Piat

Reputation: 4561

For some reason, the DNS resolver of the kube-controller-manager pod on kubernet master was empty:

docker exec  kube-controller-manager cat /etc/resolv.conf
# Generated by dhcpcd
# /etc/resolv.conf.head can replace this line
# /etc/resolv.conf.tail can replace this line

Since the host server resolv.confwas correct, I simply restarted the container:

docker restart kube-controller-manager

(an alternative ugly way would have been to edit the resolv.conf manually using docker restart kube-controller-manager, then do run the appropriate echo XXX >> /etc/resolv.conf ... bad idea )

Some other containers may have a similar issue on this node. This is a hacky way to identify and restart those containers:

cd  cd /var/lib/docker/containers
ls -1 "$(grep nameserver -L */resolv.conf)" | sed -e 's#/.*##' 
0c10e1374644cc262c8186e28787f53e02051cc75c1f943678d7aeaa00e5d450
70fd116282643406b72d9d782857bb7ec76dd85dc8a7c0a83dc7ab0e90d30966
841def818a8b4df06a0d30b0b7a66b75a3b554fb5feffe78846130cdfeb39899
ae356e26f1bf8fafe530d57d8c68a022a0ee0e13b4e177d3ad6d4e808d1b36da
d593735a2f6d96bcab3addafcfe3d44b6d839d9f3775449247bdb8801e2e1692
d9b0dfaa270d3f50169fb1aed064ca7a594583b9f71a111f653c71d704daf391

Restart affected containers:

cd /var/lib/docker/containers ; ls -1 $(grep nameserver -L */resolv.conf) | sed -e 's#/.*##' | xargs -n1 -r docker restart

Upvotes: 1

Related Questions