CODEWITHSUNDEEP
ubuntunetwork-programminglxcopenvswitchlxd
Nagri
Nagri

Reputation: 3136

Want to use the `Vlan` feature of `OpenVSwitch` with `LXD/LXC`

I want to use the Vlan feature of OpenVSwitch. This is what I have done so far;

hussain@hussain:~$ lxc list
+----------+---------+------+------+------------+-----------+
|   NAME   |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+------+------+------------+-----------+
| trusty-1 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+
| trusty-2 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+
| trusty-3 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+

and this;

hussain@hussain:~$ sudo ovs-vsctl show
a8498c25-8432-4174-9869-6eae38044cfe
    Bridge "br0"
        Controller ptcp
        Port "vport1"
            Interface "vport1"
        Port "enp1s0"
            Interface "enp1s0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.0"

In trusty-1 container I have this ;

root@trusty-1:~# cat /etc/network/interfaces.d/eth0.cfg 
#auto eth0
#iface eth0 inet static
#     address 172.16.3.250
#     netmask 255.255.254.0
#     gateway 172.16.2.1


auto eth0
iface eth0 inet static
        address 172.16.3.250
        network 172.16.0.0
        netmask 255.255.254.0
        gateway 172.16.2.1
        dns-nameservers 172.16.3.199 8.8.8.8
        dns-search google.com
        bridge_ports vport1
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

What I want to do is that trusty-1 container use vport1 bridge port so that I can tag it and then manipulate it for later future cases, but what happens instead is this;

hussain@hussain:~$ sudo ovs-vsctl show
a8498c25-8432-4174-9869-6eae38044cfe
    Bridge "br0"
        Controller ptcp
        Port "vport1"
            Interface "vport1"
        Port vethHBNLTA
            Interface vethHBNLTA
        Port "enp1s0"
            Interface "enp1s0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.0"

every time I up trusty-1 a temporary port (vethHBNLTA in this case) gets created.

How can I make sure that the container trusty-1 always uses vport1 ?

Thank you.

P.S.: So a God sent angel on the LXC IRC told me to do;

$ lxc config edit trusty-1

and add 

  eth0:
    name: eth0
    nictype: physical
    parent: vport1
    type: nic

under devices:.

so I did. But the networks connectivity went out. Couldn't ping either the host or Google from the container anymore.

SO then I changed the container config to this;

devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: vport1
    type: nic

Now I cant even start my container. The error logs shows this;

hussain@hussain:~$ lxc info --show-log trusty-1
Name: trusty-1
Architecture: x86_64
Created: 2016/08/10 06:48 UTC
Status: Stopped
Type: persistent
Profiles: default

Log:

            lxc 20160823145623.424 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3
            lxc 20160823145623.424 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.426 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:797 - Attempting to set proc title to [lxc monitor] /var/lib/lxd/containers trusty-1
            lxc 20160823145623.427 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.428 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
            lxc 20160823145623.428 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 1 start' for container 'trusty-1', config section 'lxc'
            lxc 20160823145623.429 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3
            lxc 20160823145623.429 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.431 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:178 - using monitor sock name lxc/d78a9d7e97b4b375//var/lib/lxd/containers
            lxc 20160823145623.451 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
            lxc 20160823145623.451 DEBUG    lxc_console - console.c:lxc_console_peer_default:469 - no console peer
            lxc 20160823145623.451 INFO     lxc_start - start.c:lxc_init:488 - 'trusty-1' is initialized
            lxc 20160823145623.451 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145623.451 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145623.452 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
            lxc 20160823145623.452 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
            lxc 20160823145623.487 ERROR    lxc_conf - conf.c:instantiate_veth:2595 - failed to attach 'veth1SE4RV' to the bridge 'vport1': Operation not permitted
            lxc 20160823145623.512 ERROR    lxc_conf - conf.c:lxc_create_network:2872 - failed to create netdev
            lxc 20160823145623.512 ERROR    lxc_start - start.c:lxc_spawn:1080 - failed to create the network
            lxc 20160823145623.512 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty-1'
            lxc 20160823145623.512 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty-1', config section 'lxc'
            lxc 20160823145624.015 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 1 stop' for container 'trusty-1', config section 'lxc'
            lxc 20160823145624.087 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
            lxc 20160823145624.087 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
            lxc 20160823145624.093 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145624.093 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145635.567 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145635.567 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145635.575 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145635.575 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536

In the logs there comes a line;

Failed to attach 'veth1SE4RV' to the bridge 'vport1': Operation not permitted

Does this have something to do with permissions of some sort ?

Upvotes: 0

Views: 3042

Answers (3)

Nilesh
Nilesh

Reputation: 624

Much easier solution found:

ovs-vsctl add-br vlan418 vm-bridge 418

vlan418 is name of the fake bridge to be created and vm-bridge the primary bridge which is bound to the physical NIC of the VM host. 418 is the vlan id.

Then in lxd container configuration:

devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: vlan418
    type: nic

That's it. It works flawlessly. add-br creates a fake bridge and OpenVSwitch creates the required port automatically when lxd tries to attach port to the fake bridge vlan418.

Upvotes: 0

Nagri
Nagri

Reputation: 3136

So after a lot of hit and try I somehow managed to do what I wanted. For someone who comes here in future with exactly the same problem I am leaving you this.

To connect your container with an existing interface to this:

  1. Create a bridge on the host, lets call it br0.
  2. Create a virtual interface on the host with type as internal, lets call it vport. this will do ovs-vsctl add-port br0 vport -- set interface vport type=internal

  3. do a $ lxc config edit <containername> and under device put this

    eth0: name: eth0 nictype: physical parent: vport type: nic

This is the part which tells LXC to use vport as its interface.

Thats all you'll have to do to make this work.

Though the packet drop inside the container was huge with this configuration and this made me go for the second option, which is as follows.

To let your container create its own virtual interface

  1. Create a bridge on the host, lets call it br0.

  2. do a $ lxc config edit <containername> and under device put this

    eth0: host_name: vport name: eth0 nictype: bridged parent: br0 type: nic

What this does is every time LXC restarts it creates its own interface but every time the name of the interface is vport.

Packet drops in this configuration was none so I am going with this.

Upvotes: 0

Zang MingJie
Zang MingJie

Reputation: 5275

  1. Set vport1 type to internal, by doint so, ovs will create a virtual interface vport1. Please ensure you can see the interface by using ip link show before creating the lxc container
  2. Set lxc.network.type to phys, so lxc will use the existing interface vport1 created by ovs instead of creating a new interface

Upvotes: 1

Related Questions