CODEWITHSUNDEEP
dockermicroservicesconsulhealth-monitoring
junglie85
junglie85

Reputation: 1463

Docker for Mac Container to Host Networking - Consul Health Checks Connection Refused

I have a HTTP health check in my service, exposed on localhost:35000/health. At the moment it always returns 200 OK. The configuration for the health check is done programmatically via the HTTP API rather than with a service config, but in essence, it is:

set id: service-id
set name: health check
set notes: consul does a GET to '/health' every 30 seconds
set http: http://127.0.0.1:35000/health
set interval: 30s

When I run consul in dev mode (consul agent -dev -ui) on my host machine directly the health check passes without any problem. However, when I run consul in a docker container, the health check fails with:

2017/07/08 09:33:28 [WARN] agent: http request failed 'http://127.0.0.1:35000/health': Get http://127.0.0.1:35000/health: dial tcp 127.0.0.1:35000: getsockopt: connection refused

The docker container launches consul, as far as I am aware, in exaclty the same state as the host version:

version: '2'
services:
  consul-dev:
    image: "consul:latest"
    container_name: "net-sci_discovery-service_consul-dev"
    hostname: "consul-dev"
    ports:
      - "8400:8400"
      - "8500:8500"
      - "8600:8600"
    volumes:
      - ./etc/consul.d:/etc/consul.d
    command: "agent -dev -ui -client=0.0.0.0 -config-dir=/etc/consul.d"

I'm guessing the problem is that consul is trying to do the GET request to the containers loopback interface rather than what I am intending, which is for the loopback interface of the host. Is that a correct assumption? More importantly, what do I need to do to correct the problem?

Upvotes: 0

Views: 398

Answers (1)

junglie85
junglie85

Reputation: 1463

So it transpires that there was a bug in some previous versions of macOS that prevented use of the docker0 network. Whilst the bug is fixed in newer versions, Docker support extends to older versions and so Docker for Mac doesn't currently support docker0. See this discussion for details.

The workaround is to create an alias to the loopback interface on the host machine, set the service to listen on either that alias or 0.0.0.0, and configure Consul to send the health check GET request to the alias.

To set the alias (choose a private IP address that's not being used for anything else; I chose a class A address but that's irrelevant):

sudo ifconfig lo0 alias 10.200.10.1/24

To remove the alias:

sudo ifconfig lo0 -alias 10.200.10.1

From the service definition above, the HTTP line should now read:

set http: http://10.200.10.1:35000/health

And the HTTP server listening for the health check requests also needs to be listening on either 10.200.10.2 or 0.0.0.0. This latter option is suggested in the discussion but I've only tried it with the alias.

I've updated the title of the question to more accurately reflect the problem, now I know the solution. Hope it helps somebody else too.

Upvotes: 0

Related Questions