127.0.0.1:5000: getsockopt: connection refused in Minikube

Question

Using minikube and docker on my local Ubuntu workstation I get the following error in the Minikube web UI:

Failed to pull image "localhost:5000/samples/myserver:snapshot-180717-213718-0199": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: getsockopt: connection refused

after I have created the below deployment config with:

kubectl apply -f hello-world-deployment.yaml

hello-world-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-world
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: hello-world
        tier: backend
    spec:
      containers:
      - name: hello-world
        image: localhost:5000/samples/myserver:snapshot-180717-213718-0199
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - containerPort: 8080

And output from docker images:

REPOSITORY                                 TAG                           IMAGE ID            CREATED             SIZE
samples/myserver                           latest                        aa0a1388cd88        About an hour ago   435MB
samples/myserver                           snapshot-180717-213718-0199   aa0a1388cd88        About an hour ago   435MB
k8s.gcr.io/kube-proxy-amd64                v1.10.0                       bfc21aadc7d3        3 months ago        97MB

Based on this guide: How to use local docker images with Minikube?

I have also run:

eval $(minikube docker-env)

and based on this:

https://github.com/docker/for-win/issues/624

I have added:

    "InsecureRegistry": [
        "localhost:5000",
        "127.0.0.1:5000"
    ],

to /etc/docker/daemon.json

Any suggestion on what I missing to get the image pull to work in minikube?

I have followed the steps in the below answer but when I get to this step:

$ kubectl port-forward --namespace kube-system $(kubectl get po -n kube-system | grep kube-registry-v0 | awk '{print $1;}') 5000:5000

it just hangs like this:

$ kubectl port-forward --namespace kube-system $(kubectl get po -n kube-system | grep kube-registry-v0 | awk '{print $1;}') 5000:5000
Forwarding from 127.0.0.1:5000 -> 5000
Forwarding from [::1]:5000 -> 5000

and I get the same error in minikube dashboard after I create my deploymentconfig.

Based on answer from BMitch I have now tried to create a local docker repository and push an image to it with:

$ docker run -d -p 5000:5000 --restart always --name registry registry:2
$ docker pull ubuntu
$ docker tag ubuntu localhost:5000/ubuntu:v1
$ docker push localhost:5000/ubuntu:v1

Next when I do docker images I get:

$ docker images
REPOSITORY                                      TAG                           IMAGE ID            CREATED             SIZE
ubuntu                                          latest                        74f8760a2a8b        4 days ago          82.4MB
localhost:5000/ubuntu                           v1                            74f8760a2a8b        4 days ago          82.4MB

I have then updated my deploymentconfig hello-world-deployment.yaml to:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-world
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: hello-world
        tier: backend
    spec:
      containers:
      - name: hello-world
        image: localhost:5000/ubuntu:v1
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - containerPort: 8080

and

kubectl create -f hello-world-deployment.yaml

But in Minikube I still get similar error:

 Failed to pull image "localhost:5000/ubuntu:v1": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: getsockopt: connection refused 

So seems Minikube is not allowed to see the local registry I just created?

Answer 1

it is unclear from your question how many nodes do you have?

If you have more than one, your problem is in your deployment with replicas: 1.

If not, please ignore this answer.

You don't know where and what that replica will be. So if you don't have docker local registry on all of your nodes, and you got unlucky that kubernetes is trying to use some node without docker registry, you will end up with that error.

Same thing happened to me, same error connection refused because deployment went to node without local docker registry.

As I am typing this, I think this can be resolved with ingress. You do registry as deployment, add service, add volume for images and put it to ingress.

Little more of work but at least all your nodes will be sync (all of your pods sorry).

Answer 2

If you're specifying the image source as the local registry server, you'll need to run a registry server there, and push your images to it.

You can self host a registry server with multiple 3rd party options, or run this one that is packaged inside a docker container: https://hub.docker.com/_/registry/

This only works on a single node environment unless you setup TLS keys, trust the CA, or tell all other nodes of the additional insecure registry.

You can also specify the imagePullPolicy as Never.

---

Both of these solutions were already in your linked question and I'm not seeing any evidence of you trying either in this question. Without showing how you tried those steps and experienced a different problem, this question should probably be closed as a duplicate.

Answer 3

It looks like you’re facing a problem with localhost on your computer and localhost used within the context of minikube VM. To have registry working, you have to set an additional port forwarding.

If your minikube installation is currently broken due to a lot of attempts to fix registry problems, I would suggest restarting minikube environment:

minikube stop && minikube delete && rm -fr $HOME/.minikube &&  minikube start

Next, get kube registry yaml file:

curl -O https://gist.githubusercontent.com/coco98/b750b3debc6d517308596c248daf3bb1/raw/6efc11eb8c2dce167ba0a5e557833cc4ff38fa7c/kube-registry.yaml

Then, apply it on minikube:

kubectl create -f kube-registry.yaml

Test if registry inside minikube VM works:

minikube ssh && curl localhost:5000

On Ubuntu, forward ports to reach registry at port 5000:

kubectl port-forward --namespace kube-system $(kubectl get po -n kube-system | grep kube-registry-v0 | awk '{print $1;}') 5000:5000

If you would like to share your private registry from your machine, you may be interested in sharing local registry for minikube blog entry.