Local private Docker registry will not respond

Question

I spent quite a bit of time trying to get a swarm up and running. Actually, getting it running was a breeze using the script provided in this article.

But I then ran into an issue where the workers didn't have the same images as the master. It seems as though this shouldn't be an issue because each worker node is launched with the --registry-mirror flag.

But as I tried to debug this, I found that I seem to have something fundamentally wrong somewhere. So before dealing with the more complicated ins and outs of a swarm registry, I need to be able to just run a basic local registry and I'm running into problems there.

The Docker documentation talks about running your own registry on localhost. So with absolutely nothing else running on my Docker host and following that guide, I did:

$ docker run -d -p 5000:5000 --restart=always --name registry registry:2

Running docker ps shows it launched just fine. Grabbing the latest ubuntu image also works without issue:

$ docker pull ubuntu && docker tag ubuntu localhost:5000/ubuntu

But the next step is where the problem is:

$ docker push localhost:5000/ubuntu
The push refers to a repository [localhost:5000/ubuntu]
Get https://localhost:5000/v1/_ping: dial tcp 192.168.21.146:5000: i/o timeout

Now I'm using Docker for Mac (Docker version 17.03.1-ce, build c6d412e) and I have configured localhost:5000 to be an insecure registry in the preferences (and restarted). So I'm absolutely baffled why a) it brings up 192.168.21.146? (that is my subnet, but not my host's address) and b) it references https?

I thought perhaps something was wrong with the DNS resolution so I checked /etc/hosts and I even ran this script in node just to double check:

$ node
> var x = require('dns').lookup("localhost", (err, ip) => console.log("IP = ", ip))
undefined
> IP =  127.0.0.1

So it doesn't seem to be an issue with incorrect resolution of localhost. This is further substantiated by the fact that if I run curl http://localhost:5000/v2/_catalog, I get an empty registry, as I would expect. So why is the push failing? However, if I do this:

$ docker pull ubuntu && docker tag ubuntu 127.0.0.1:5000/ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
Digest: sha256:f3a61450ae43896c4332bda5e78b453f4a93179045f20c8181043b26b5e79028
Status: Image is up to date for ubuntu:latest

$ docker push 127.0.0.1:5000/ubuntu
The push refers to a repository [127.0.0.1:5000/ubuntu]
73e5d2de6e3e: Pushed
08f405d988e4: Pushed
511ddc11cf68: Pushed
a1a54d352248: Pushed
9d3227c1793b: Pushed
latest: digest: sha256:f3a61450ae43896c4332bda5e78b453f4a93179045f20c8181043b26b5e79028 size: 1357

Now I'm totally confused. Why should 127.0.0.1 work when localhost doesn't?

I've got other questions about setting up a local registry and mirroring that in a swarm to worker nodes. But I don't think there is any sense in asking about the problems I'm having there until I can at least get through the 4 basic steps in the Docker documentation.

Are there some diagnostics I can run here (that I haven't already) to figure out what is misconfigured?!? Any help in diagnosing this would be greatly appreciated. Thanks.

Answer 1

So after writing all this up, I dug even deeper. I see that somehow the IP address 192.168.21.146 was popping up. Given all the things that Docker does with networking, I couldn't be sure it wasn't somehow grabbing an IP from my network. But it turned out that when I went into my router to figure out what machine was 192.168.21.146, I found out not only that it was a TV, but that the TV was actually registering itself in the DNS with the hostname localhost!!!

It seems that Docker must use a different resolution method than pretty much every other program on my Mac which is why I got thrown off the scent. But, I decided to keep the post here because it is possible somebody else (who, for example, owns a Samsung smart TV) might run into this same issue.

Ugh.