Cannot add a DPDK-bound interface to the OVS bridge

Question

I am unsucessfully trying to add a PCI interface (I219-V) to the OVS bridge using ovs-vsctl:

root@pc:/home/me# ovs-vsctl add-port br0 tryport -- set Interface tryport type=dpdk options:dpdk-devargs=0000:00:1f.6
ovs-vsctl: Error detected while setting up 'tryport': Error attaching device '0000:00:1f.6' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".

OVS and DPDK releases:

ovs-vsctl: 2.15.90
DB Schema 8.2.0
libdpdk: 21.02.0

The addition fails with:

2021-02-19T18:12:32.543Z|00126|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.543Z|00127|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.543Z|00128|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.543Z|00129|netdev|WARN|tryport: could not set configuration (Invalid argument)
2021-02-19T18:12:32.543Z|00130|dpdk|ERR|Invalid port_id=32
2021-02-19T18:12:32.547Z|00131|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.547Z|00132|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.547Z|00133|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.547Z|00134|netdev|WARN|tryport2: could not set configuration (Invalid argument)
2021-02-19T18:12:32.547Z|00135|dpdk|ERR|Invalid port_id=32

This is the content of the OVS DB, EAL seems to initialize correctly:

root@pc:/home/me# ovs-vsctl list Open_Vswitch
_uuid               : 4150fd20-ecbd-4ee3-a438-0c74d0411831
bridges             : []
cur_cfg             : 9
datapath_types      : [netdev, system]
datapaths           : {}
db_version          : "8.2.0"
dpdk_initialized    : true
dpdk_version        : "DPDK 21.02.0"
external_ids        : {hostname=pc, rundir="/usr/local/var/run/openvswitch", system-id=""}
iface_types         : [bareudp, dpdk, dpdkvhostuser, dpdkvhostuserclient, erspan, geneve, gre, gtpu, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 9
other_config        : {dpdk-init="true"}
ovs_version         : "2.15.90"
ssl                 : []
statistics          : {}
system_type         : Ubuntu
system_version      : "20.04-focal"

This is the current setup of interfaces, the device is bound to the vfio-pci using dpdk-devbind:

➜  ~ dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:00:1f.6 'Ethernet Connection (10) I219-V 0d4f' drv=vfio-pci unused=

Network devices using kernel driver
===================================
0000:00:14.3 'Wireless-AC 9462 02f0' if=wlp0s20f3 drv=iwlwifi unused=vfio-pci *Active*

IOMMU is enabled:

➜  ~ sudo dmesg | grep -e DMAR -e IOMMU
[    0.015758] ACPI: DMAR 0x000000006CAFE000 0000A8 (v01 LENOVO TP-N2Q   00001250 PTEC 00000002)
[    0.055802] DMAR: IOMMU enabled
[    0.157186] DMAR: Host address width 39
[    0.157188] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.157196] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.157198] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.157203] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.157205] DMAR: RMRR base: 0x0000006e81a000 end: 0x0000006e839fff
[    0.157207] DMAR: RMRR base: 0x00000079000000 end: 0x0000007d7fffff
[    0.157210] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.157212] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.157213] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.159739] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.890022] DMAR: No ATSR found
[    0.890108] DMAR: dmar0: Using Queued invalidation
[    0.890113] DMAR: dmar1: Using Queued invalidation
[    0.891306] DMAR: Intel(R) Virtualization Technology for Directed I/O

My grub and hugepages settings:

➜  ~ cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.10.7-051007-generic root=UUID=455abfd8-5778-4930-8062-0ba7a48be98e ro quiet splash iommu=pt intel_iommu=on vt.handoff=7

grep HugePages_ /proc/meminfo
HugePages_Total:    2048
HugePages_Free:     1536
HugePages_Rsvd:        0
HugePages_Surp:        0

Any hints on how to debug the issue further? The EAL error report "Driver cannot attach the device" is unfortunately not very conclusive.

Cannot add a DPDK-bound interface to the OVS bridge

Answers (1)

Related Questions