CODEWITHSUNDEEP
linuxdockerubuntunetwork-programming
Yue Wang
Yue Wang

Reputation: 21

Unable to apply tc filter on source or destination address of docker containers

I am trying to use the tcconfig package (a python wrapper for tc module) and tc commands to set up netem delay between 2 docker containers within a Ubuntu distro in WSL2

However I didn't manage to make it work as the tc commands can not be applied to the correct address.

What I have tried is:

# create and start 3 docker containers:
sudo docker run --name dc1 -itd --cap-add NET_ADMIN archlinux bash
sudo docker run --name dc2 -itd --cap-add NET_ADMIN archlinux bash
sudo docker run --name dc3 -itd --cap-add NET_ADMIN archlinux bash

# use tcconfig to set up 100ms delay between container dc1 and dc2
sudo tcset dc1 --src-container dc1 --dst-container dc2 --docker --delay 100ms

With the above commands I expect to be able to see 100ms delay when I ping dc1 from dc2 and vice versa.

However I cannot see the delay applied.

When I look into the veth of container dc1, I can see the following rules applied with the tcshow command from tcconfig:

(.venv) johnsmith@AP-HYB-AmXNp8wl:~/python-scripts$ sudo tcshow veth605f18a
{
    "veth605f18a": {
        "outgoing": {
            "src-network=172.17.0.2/32, dst-network=172.17.0.3/32, protocol=ip": {
                "filter_id": "800::800",
                "delay": "100ms",
                "rate": "10Gbps"
            }
        },
        "incoming": {}
    }
}

I figure the reason that it's not working might be because the rules is somehow applied to 172.17.0.2/32 and 172.17.0.3/32 instead of 172.17.0.2/16 and 172.17.0.3/16, which is the correct address + subnet of the containers. To validate the address of container dc1:

[root@7cee5e6f43bc /]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
11: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet **172.17.0.2/16** brd 172.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever

If I dive deeper into the tc commands that are behind the scene, I can get the following commands with the --tc-script option:

/usr/sbin/tc qdisc add dev veth605f18a root handle 17e0: htb default 1
/usr/sbin/tc class add dev veth605f18a parent 17e0: classid 17e0:1 htb rate 10000000.0kbit
/usr/sbin/tc class add dev veth605f18a parent 17e0: classid 17e0:76 htb rate 10000000.0Kbit ceil 10000000.0Kbit
/usr/sbin/tc qdisc add dev veth605f18a parent 17e0:76 handle 22d8: netem delay 100.0ms
/usr/sbin/tc filter add dev veth605f18a protocol ip parent 17e0: prio 5 u32 match ip src 172.17.0.2/32 match ip dst 172.17.0.3/32 flowid 17e0:76

As I thought the problem was the subnet mask /32 not being correct, I attempted to run the above tc commands as follow, with only the subnet masks updated to /16:

/usr/sbin/tc qdisc add dev veth605f18a root handle 17e0: htb default 1
/usr/sbin/tc class add dev veth605f18a parent 17e0: classid 17e0:1 htb rate 10000000.0kbit
/usr/sbin/tc class add dev veth605f18a parent 17e0: classid 17e0:76 htb rate 10000000.0Kbit ceil 10000000.0Kbit
/usr/sbin/tc qdisc add dev veth605f18a parent 17e0:76 handle 22d8: netem delay 100.0ms
/usr/sbin/tc filter add dev veth605f18a protocol ip parent 17e0: prio 5 u32 match ip src 172.17.0.2/16 match ip dst 172.17.0.3/16 flowid 17e0:76

I still fail to make this work as I found out the set of tc rules eventually set the filters on the entire subnet of 172.17.0.0/16 instead of these 2 specific ip addresses:

(.venv) johnsmith@AP-HYB-AmXNp8wl:~/python-scripts$ tcshow veth605f18a 
{
    "veth605f18a": {
        "outgoing": {
            "src-network=172.17.0.0/16, dst-network=172.17.0.0/16, protocol=ip": {
                "filter_id": "800::800",
                "delay": "100ms",
                "rate": "10Gbps"
            }
        },
        "incoming": {}
    }
}

I am very new to network shaping with tc, does anyone have some idea of what did I do wrong here and how can I apply the delay between the docker contaienrs? Many thanks!

Upvotes: 0

Views: 360

Answers (1)

Yue Wang
Yue Wang

Reputation: 21

After some struggling I managed to find out what was going wrong: I didn't set the src-container and dst-container correctly:

The correct command with tcconfig to set up delay between container dc1 and dc2 should be:

sudo tcset dc1 --src-container dc2 --dst-container dc1 --docker --delay 100ms

instead of

sudo tcset dc1 --src-container dc1 --dst-container dc2 --docker --delay 100ms

Incorrect assignment of src/dst container will result in a false tc filter getting applied to the qdisc:

/usr/sbin/tc qdisc add dev vethf83016b root handle 1da0: htb default 1
/usr/sbin/tc class add dev vethf83016b parent 1da0: classid 1da0:1 htb rate 10000000.0kbit
/usr/sbin/tc class add dev vethf83016b parent 1da0: classid 1da0:192 htb rate 10000000.0Kbit ceil 10000000.0Kbit
/usr/sbin/tc qdisc add dev vethf83016b parent 1da0:192 handle 2c9e: netem delay 200.0ms 10ms loss 20% 10%
/usr/sbin/tc filter add dev vethf83016b protocol ip parent 1da0: prio 5 u32 match ip dst 172.17.0.3 match ip src 172.17.0.2 flowid 1da0:192

In the case dc1 has the virtual interface of vethf83016b and ip address of 172.17.0.2 While dc2 has an ip address of 172.17.0.3

From my understanding, with the above tc settings, the virtual ethernet of dc1 should be able to apply the delay to all the packets sent to dc2. However, it only works the other way around when I set it up with dc1 as destination and dc2 as source where a simulated 2-way delay can be achieved:

  • when I ping dc2 from dc1: the response packet will be delayed.
  • when I ping dc1 from dc2: the request packet will be delayed.

However I don't fully understand how the veth work with the outgoing traffic, does anyone have any idea why is

sudo tcset dc1 --src-container dc1 --dst-container dc2 --docker --delay 100ms

not delaying the outgoing packets from dc1 to dc2?

Upvotes: 1

Related Questions