CODEWITHSUNDEEP
dockerdocker-composedocker-network
orshachar
orshachar

Reputation: 5037

no internet inside docker-compose service

I cannot reach external network from docker-compose containers.

Consider the following docker-compose file:

version: '2'
services:
    nginx:
      image: nginx

Using the simple docker run -it nginx bash I manage to reach external IPs or Internet IPs (ping www.google.com).

On the other hand if I use docker-compose and attach to the container, I cannot reach external IP addresses / DNS.

docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 7
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-38-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.859 GiB
Name: ***
ID: ****
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
 127.0.0.0/8

docker-compose 1.8.1, build 878cff1

daemon.json file:

{
  "iptables" : false,
  "dns" : ["8.8.8.8","8.8.4.4"]
}

Upvotes: 46

Views: 42101

Answers (6)

Dmitrij Kultasev
Dmitrij Kultasev

Reputation: 5745

In my case I had docker installed via snap, after removing it and installing by following instructions from the official website it started to work normally.

Upvotes: 0

shmeeps
shmeeps

Reputation: 7833

I just ran into this. Running any container with docker run -it <image> bash, I was able to do anything network related that I needed, but attaching to a container started through docker-compose with docker exec -it <conatiner> bash could run nothing. Even apt-get would fail with a "temporary failure in name resolution error."

I found that adding network_mode: "bridge" to each service would enable external networking in each container, at the expense of attaching everything to the default docker network. I wanted to keep the stack isolated though, so set about an alternative solution.

I ran docker inspect network bridge to view the default network, and the same with my stack network. The only significant differences I found were a difference in subnet/gateway (which I believe is to be expected, however the IPs listed differed significantly, 192.* vs 172.*), and that the default network had several options that were not present in the stack network:

"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"

I was able to isolate it to "com.docker.network.bridge.default_bridge": "true", however this broke networking for containers outside the stack, to the point where I eventually had to completely remove and reinstall docker. I'm assuming the docker daemon gets confused with two networks marked as default and is unable to resolve it, even after removing the stack network.

One thing I noticed after doing that though was that the subnet and getway were just off by one from the default adapter. I verified that removing the above option reverted the subnet and gateway to what I saw originally, which it did. I then used this network config in my compose file:

networks:
  default:
    driver: "bridge"
    ipam:
      driver: default
      config:
        - subnet: X.X.0.0/16

Where the first value of the subnet was the same as the default bridge adapater, and the second value was plus one. And that at least got everything working fine, sans the default adapter still being broken.

Interestingly enough, when I went to reinstall docker to try and repair the default network, I basically ran the following after looking at the docker compose network documentation for external networking.

sudo snap remove docker
sudo sysctl net.ipv4.conf.all.forwarding=1
sudo iptables -P FORWARD ACCEPT
sudo snap install docker

And a subsequent docker-compose up -d with the network settings removed resulted in the same subnet IP that I had defined earlier without having the config present for it. I believe the underlying issue is simply that something was preventing the docker daemon from properly finding an available, valid subnet, and once that is resolved everything should again work out of the box.

Upvotes: 3

peedee
peedee

Reputation: 3739

The last time I had a problem like that, I solved it like this:

https://github.com/docker/docker/issues/866#issuecomment-19218300

pkill docker
iptables -t nat -F
ifconfig docker0 down
brctl delbr docker0
docker -d

It will force docker to recreate the bridge and reinit all the network rules.

As for reasons why this happens, I don't have good answers. But I did recently trace the problem back to journald. When I restart journald (for example because I changed its config), DNS resolution inside docker-compose containers consistently/reproducibly breaks. I don't know why exactly, I can only say that this is a reliable way for me to reproduce it on RHEL.

EDIT The docker -d command might not work for you based on the version of docker you are using but don't worry about it, you can omit that command.

Upvotes: 9

cornernote
cornernote

Reputation: 1095

Check /etc/default/docker to ensure it doesn't have the following line:

DOCKER_OPTS="--iptables=false"

Also check /etc/docker/daemon.json to ensure it doesn't have the following key:

{
"iptables":false
}

We added this on one server to get UFW working with docker. We then changed to an external firewall. Spent ages looking for the reason external networking wasn't working because it was removed from our deploy guide. Hope this helps someone else.

Upvotes: 10

Bruno Borges
Bruno Borges

Reputation: 851

The image nginx you are pulling doesn't have ping installed by default. So if you are really using ping to test your connection, you must first install it.

I created a custom Dockerfile to have it installed:

FROM nginx:latest
RUN apt update && apt -y install iputils-ping

Then I built it locally and tagged as mynginx.

Then I changed docker-compose.yml to use the custom image mynginx:

version: '2'
services:
  nginx:
    image: mynginx

Finally, I fired docker-compose up, and did a docker exec into it, and tested ping. All worked just fine. I also did docker run -ti ... and it worked.

What bugs me on your question is how docker run can give you a container with different behaviour than if created by docker-compose up.

More clarity on how you check internet access would be helpful though.

Upvotes: 0

Gao Shenghan
Gao Shenghan

Reputation: 446

Docker containers has the ability to access internet by default. Here is how I solved the problem last week: docker container can only access internet with net host

Or you just let the container in host mode:

version: '2'
service:
  nginx:
    image: nginx
    network_mode: host

But as @peedee pointed out on comment, this solution will lost network separation between host and containers.

Upvotes: 0

Related Questions