CODEWITHSUNDEEP
network-programmingdockerdocker-composenic
tgogos
tgogos

Reputation: 25152

docker-compose: specify which interface will connect to each network (when having multiple networks)

Inside the docker-compose.yaml I define 4 different networks:

networks:
  vpp-nw:
  vpp-client:
  vpp-server:
  vpp-db:

which use the following network addresses respectively:

172.20.0.x
172.21.0.x
172.22.0.x
172.23.0.x

One of the containers I use, connects to all 4 networks (with the same order):

# part of docker-compose.yaml

services:
  my_tool:
    build: ./my_tool
    image: tgogos/my_tool
    container_name: my_tool_docker_comp
    hostname: my_tool
    privileged: true
    links:
      - my_db
    networks:
      - vpp-nw
      - vpp-client
      - vpp-server
      - vpp-db
    ports:
      - "8888:8888"

Is there a way to define which interface will connect to each network? For example, I would like:

Below you can see an ifconfig output of this container. NICs seem to connect to each network in an arbitrary way every time I docker-compose down | docker-compose up...

eth0  Link encap:Ethernet  HWaddr 02:42:ac:15:00:03  
      inet addr:172.21.0.3  Bcast:0.0.0.0  Mask:255.255.0.0

eth1  Link encap:Ethernet  HWaddr 02:42:ac:17:00:03  
      inet addr:172.23.0.3  Bcast:0.0.0.0  Mask:255.255.0.0

eth2  Link encap:Ethernet  HWaddr 02:42:ac:14:00:02  
      inet addr:172.20.0.2  Bcast:0.0.0.0  Mask:255.255.0.0

eth3  Link encap:Ethernet  HWaddr 02:42:ac:16:00:03  
      inet addr:172.22.0.3  Bcast:0.0.0.0  Mask:255.255.0.0

Edit:

Further reading, github issues:

Upvotes: 24

Views: 26831

Answers (3)

tu4n
tu4n

Reputation: 4450

You can use the priority attribute in Docker compose reference to specify the order which compose attach the networks to your container.

https://docs.docker.com/compose/compose-file/compose-file-v2/#priority

...

services:
  foo:
    image: foo:latest
    restart: always
    networks:
      network1:
        priority: 1000 # eth0
      network2:
        priority: 900 # eth1
      network3: # Default priority is 0, eth2

...

Upvotes: 3

itiic
itiic

Reputation: 3712

You can try to mount config files

i.e. for RH based images:

/etc/sysconfig/network-scripts/ifcfg-eth0

So, your yml file would look like below:

# part of docker-compose.yaml

services:
  my_tool:
    build: ./my_tool
    image: tgogos/my_tool
    container_name: my_tool_docker_comp
    hostname: my_tool
    privileged: true
    links:
      - my_db
    networks:
      - vpp-nw
      - vpp-client
      - vpp-server
      - vpp-db
    volumes:
     - ./conf/eth0:/etc/sysconfig/network-scripts/ifcfg-eth0
     - ./conf/eth1:/etc/sysconfig/network-scripts/ifcfg-eth1
     - ./conf/eth2:/etc/sysconfig/network-scripts/ifcfg-eth2
     - ./conf/eth3:/etc/sysconfig/network-scripts/ifcfg-eth3
    ports:
      - "8888:8888"

Upvotes: 1

larsks
larsks

Reputation: 311712

This isn't an answer, but it is a confirmation of the behavior you've reported along with complete reproducers that could be used to file a bug report.

I am able to reproduce your behavior. I think it's a bug in docker-compose, since the networks key on your service is an ordered list. I would expect networks to be assigned to interfaces in the order in which they are listed in the file.

Using this docker-compose.yml (in a directory named nwtest):

version: "2"

services:
  server:
    image: alpine
    command: sleep 999
    networks:
      - nw0
      - nw1
      - nw2
      - nw3

networks:
  nw0:
  nw1:
  nw2:
  nw3:

And this shell script:

#!/bin/sh

docker-compose up -d

for nw in 0 1 2 3; do
    nw_cidr=$(docker network inspect -f '{{ (index .IPAM.Config 0).Subnet }}' \
        nwtest_nw${nw})
    if_cidr=$(docker exec -it nwtest_server_1 ip addr show eth${nw} |
        awk '$1 == "inet" {print $2}')

    nw_net=$(ipcalc -n $nw_cidr | cut -f2 -d=)
    if_net=$(ipcalc -n $if_cidr | cut -f2 -d=)

    echo "nw${nw} $nw_net eth${nw} ${if_net}"

    if [ "$if_net" != "$nw_net" ]; then
        echo "MISMATCH: nw${nw} = $nw_net, eth${nw} = $if_net" >&2
    fi
done

docker-compose stop

You can quickly verify the problem:

$ sh runtest.sh 
Starting nwtest_server_1
nw0 192.168.32.0 eth0 192.168.32.0
nw1 192.168.48.0 eth1 192.168.48.0
nw2 192.168.64.0 eth2 192.168.80.0
MISMATCH: nw2 = 192.168.64.0, eth2 = 192.168.80.0
nw3 192.168.80.0 eth3 192.168.64.0
MISMATCH: nw3 = 192.168.80.0, eth3 = 192.168.64.0
Stopping nwtest_server_1 ...

Furthermore, this problem seems to be specific to docker-compose; if you create a Docker container with docker run and then attach multiple networks, they are always assigned to interfaces sequentially as you would expect. The test script for that is:

#!/bin/sh

docker rm -f nwtest_server_1
docker run -d --name nwtest_server_1 --network nwtest_nw0 \
    alpine sleep 999

for nw in 1 2 3; do
    docker network connect nwtest_nw${nw} nwtest_server_1
done

for nw in 0 1 2 3; do
    nw_cidr=$(docker network inspect -f '{{ (index .IPAM.Config 0).Subnet }}' \
        nwtest_nw${nw})
    if_cidr=$(docker exec -it nwtest_server_1 ip addr show eth${nw} |
        awk '$1 == "inet" {print $2}')

    nw_net=$(ipcalc -n $nw_cidr | cut -f2 -d=)
    if_net=$(ipcalc -n $if_cidr | cut -f2 -d=)

    echo "nw${nw} $nw_net eth${nw} ${if_net}"

    if [ "$if_net" != "$nw_net" ]; then
        echo "MISMATCH: nw${nw} = $nw_net, eth${nw} = $if_net" >&2
    fi
done

docker rm -f nwtest_server_1

Upvotes: 5

Related Questions