CODEWITHSUNDEEP
dpdk
kruztw
kruztw

Reputation: 23

Does the performance decrease when using DPDK with bifurcated driver or VFIO?

I learned that DPDK can work with RDMA with bifurcated driver and VFIO, but I wonder whether the performance is as good as DPDK without using bifuracated driver and VFIO.

Upvotes: 1

Views: 458

Answers (1)

maxschlepzig
maxschlepzig

Reputation: 39085

Since the OFED rdma_* API also by-passes the kernel and is on a similar low-level as the DPDK API, there aren't many possibilities for a DPDK PMD that uses it to waste performance (such as the bifurcated MLX5 PMD).

Nvidia published some performance test results for DPDK 20.05 that look fine. For example, l3fwd between 2 Mlx5 25 GBit ports (Table 6, page 14):

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Frame Size (Bytes)   Frame Rate (Mpps)   Line Rate [50G] (Mpps)   % Line Rate
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
  64                 74.40               74.40                    100.00
 128                 42.23               42.23                    100.00
 256                 22.64               22.64                    100.00
 512                 11.75               11.75                    100.00
1024                  5.99                5.99                    100.00
1280                  4.81                4.81                    100.00
1518                  4.06                4.06                    100.00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

So this is a good indication for the Mellanox bifurcated driver/PMD having similar good performance than other 'full' drivers, in general.

NB: Perhaps surprisingly, the main point of the OFED rdma_* API is kernel-bypass and not RDMA - despite its name. Meaning that you can use large parts of that API without ever doing RDMA.

A similar point can be made for the Solarflare SFC PMD which sits on top of Solarflare's ef_vi kernel bypass library. Meaning that it also uses a 'virtual interface' (vi).

PMD's that take over the whole NIC (such as the Intel ones, e.g. ixgbe) work with VFIO in order to have IOMMU protection. In fact, the DPDK guide recommends the use of VFIO with such NICs (where available) without mentioning performance caveats.

Technically, a IOMMU could add some overhead - but a good implementation doesn't have to - modulo bugs.

Upvotes: 1

Related Questions