CODEWITHSUNDEEP
kuberneteskubectl
Alan H0
Alan H0

Reputation: 81

Cannot shell into the container, rpc error: code = 5 desc ... shim-log.json: no such file or directory

trying to shell into the container by kubectl exec -it xxxxxx

but it returns 

rpc error: code = 5 desc = open /var/run/docker/libcontainerd/containerd/faf3fd49262cc738e16368001eba5e1113abcb8a87e7b818cb84af3799906149/30fe901c16e0465aa15b596bf3e4f244fb12a7e4133b6e4da5aa35167a8dfb30/shim-log.json: no such file or directory

trying to reboot the node but not help

Upvotes: 1

Views: 1335

Answers (3)

Prafull Ladha
Prafull Ladha

Reputation: 13449

The problem is with containerd, Once the containerd restart in the background, the docker daemon still try to process event streams against the old socket handles. After that, the error handeling when client can't connect to the containerd leads to the CPU spike on machine.

This is the open issue with docker and currently the workaround is to restart the docker.

sudo systemctl restart docker

Upvotes: 1

Alan H0
Alan H0

Reputation: 81

Thanks @Prafull Ladha

Eventually I restarted the docker (systemctl restart docker) of that Node which my pods could not be shelled, and it resumes to normal

Upvotes: 1

P Ekambaram
P Ekambaram

Reputation: 17679

It appears like some issue with the docker daemon. it would help if you add the logs from the container to research the root cause.

deploy alpine pod and see if you can get into the container. This is to isolate if the problem is with the platform or the pod that you are running.

kubectl run pingpong --image alpine ping 8.8.8.8
kubectl exec -it <pingpong-pod-name>  sh

most likely something wrong with the pod that you are running. share the container logs for further help

Upvotes: 0

Related Questions