CODEWITHSUNDEEP
linuxdockerdocker-composedocker-networklinux-namespaces
ArdalNelson
ArdalNelson

Reputation: 31

How to create the docker0 bridge in another network namespace then the default one?

On a server I have an environment with different network namespaces that are created with the command sudo ip netns add ns_name. The physical interfaces of the server get assigned to different namespaces, in the "default" network namespace remains no physical interface.

On startup the docker daemon creates the docker0 bridge in the "default" network namespace. How to tell docker to create the docker0 bridge not in the default one but in one of the ones created by me?

Upvotes: 3

Views: 821

Answers (1)

Phles
Phles

Reputation: 337

My system is a (2023) recent install of Archlinux, but these instructions should be similar to your distribution:

Make sure docker is stopped when you start this:

systemctl stop docker.socket
systemctl stop docker.service

Edit your docker systemd unit file, you can find it via systemctl status docker.service

● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; preset: disabled)

Modify the [Service] part of the unit file /usr/lib/systemd/system/docker.service:

[Service]
Type=notify
# This tells systemd to run dockerd in the network namespace
NetworkNamespacePath=/run/netns/ns_name
# This is the original docker command, it caused some issues for me
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
# This is the replacement if the default exec doesn't work
ExecStart=/usr/bin/dockerd -H fd:// 
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

Now refresh the configs and start docker with

systemctl daemon-reload
systemctl restart docker

If you're lucky, you can now try running a docker container and it will just work. There's a bit of weird shenanigans going on so you might get an error whenever you start any container like the one below:

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/sbin/tini": stat /sbin/tini: no such file or directory: unknown

This error appears to be from a misconfiguration between containerd and docker. Open the containerd unit file and replace its ExecStart with the following:

ExecStart=/usr/bin/containerd --config=/var/run/docker/containerd/containerd.toml

This will tell containerd to use docker's configuration and your containers should work after you reload the configs restart all the services. Frankly, it is easier to just reboot the machine at this point.

As for what this does, we get an interesting series of effects.

  • The docker executable talks to dockerd via a unix socket, so it does not matter what namespace you run docker or docker-compose commands from, as long as the socket is accessible.
  • containerd is running in the default network namespace, which has all of the proper cgroups to create containers.
  • dockerd is running inside your namespace, so it will create all network interfaces inside this namespace and all container networks will be generated inside this namespace by default.

As a final note, unless you setup a service to create your network namespace on boot, you will probably want to leave docker disabled and have a startup script create the namespace and then start docker. In addition, you may need to run chattr +i on your unit files to prevent updates from overwriting your config changes.

Upvotes: 1

Related Questions