How to compile a DPDK application as a library

Question

I have a program using DPDK and I am compiling it using the Makefile provided in the examples.

If i compile the program as an APP (as describe here), all goes well. However, my code is part of a larger project, for which the use of a separate makefile causes a lot of troubles. So I bundled my code in a library, as described in the same page.

The program that calls the functions in the library (to initialize the EAL) is getting this error:

MBUF: error setting mempool handler
Cannot init mbuf pool

It seems that, when the application is compiled as a library, the EAL cannot be initialized correctly.

I report here the steps to reproduce the problem using the l2fwd example.

Background

I built DPDK from source as described here, and I have one ethernet interface bound to DPDK drivers:

$ $RTE_SDK/usertools/dpdk-devbind.py --status |head -n4
Network devices using DPDK-compatible driver
============================================
0000:01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=igb_uio unused=ixgbe

Run the l2fwd example

Copy the example folder first and then run the example:

$ cp -r $RTE_SDK/examples/l2fwd $RTE_SDK/examples/l2fwd-lib/
$ cd $RTE_SDK/examples/l2fwd

$ make
CC main.o
LD l2fwd
INSTALL-APP l2fwd
INSTALL-MAP l2fwd.map

$ sudo ./build/l2fwd -l 0-3  -- -p 0x1
Port statistics ====================================
Statistics for port 0 ------------------------------
Packets sent:                        0
Packets received:                    0
Packets dropped:                     0
Aggregate statistics ===============================
Total packets sent:                  0
Total packets received:              0
Total packets dropped:               0
====================================================

• It works!

Build the same example as a library

• cd ../l2fwd-lib/
  mv main.c l2fwd.c
  

• Modify l2fwd.c, adding #include "l2fwd.h" on top and replacing

  int main(int argc, char **argv)
  

  with

  int start(int argc, char **argv)
  

• Create the header file l2fwd.h with the library interface:

  int start(int argc, char **argv);
  

• Modify the Makefile as described in the docs:

  APP = l2fwd      --->   LIB = libl2fwd.a
  SRCS-y := main.c --->   SRCS-y := l2fwd.c
  include $(RTE_SDK)/mk/rte.extapp.mk   --->   include $(RTE_SDK)/mk/rte.extlib.mk 
  

• Compile the library:

  $ make
  CC l2fwd.o
  AR libl2fwd.a
  INSTALL-LIB libl2fwd.a
  

• Write a program that uses the library. Create the main.c file with just these 2 lines:

  #include "l2fwd.h"
  int main (int argc, char **argv) { start(argc, argv); }
  

• compile it (with all the needed libraries):

  gcc -L build/lib/ -L $RTE_SDK/build/lib/ main.c -o main.o -l l2fwd -l dpdk -l numa -pthread -l dl 
  

• Finally, run it with the same parameters used earlier:

  $ sudo ./main.o -l 0-3  -- -p 0x1
  EAL: Detected 40 lcore(s)
  EAL: Detected 2 NUMA nodes
  EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
  EAL: Probing VFIO support...
  MAC updating enabled
  EAL: Error - exiting with code: 1
    Cause: No Ethernet ports - bye
  

In this case, the EAL cannot be initialized properly (the ethernet port is still bound to the DPDK drivers).

Edit 1

According to @Andriy Berestovskyy, the linker --whole-archive option is needed when linking DPDK libraries. This solves the problem with the examples. However, my program is facing a different problem now. I need to use a custom build system, so I am linking the DPDK application as a library. During execution I get the error:

MBUF: error setting mempool handler
mempool/dpaa2: Not a valid dpaa2 buffer pool

Looks like it is using the dpaa2 driver, that is the wrong driver (my NIC is using igb_uio). Any hints on why this is happening? The same code worked when compiled as a DPDK app, so it is probably related to the linking process.

Edit 2

This error is due to the fact that I compiled DPDK with the CONFIG_RTE_BUILD_SHARED_LIB=y option. When DPDK is built as a shared library, the driver must be explicitly set using the -d EAL command line option. I re-compiled DPDK with CONFIG_RTE_BUILD_SHARED_LIB=n and the problem was solved.

Answer 1

I got the similar issue and as mentioned in here, I used -d option to link the libray "-d /usr/lib64/librte_mempool_ring.so". It worked.

Answer 2

compile it (with all the needed libraries):

gcc -L build/lib/ -L $RTE_SDK/build/lib/ main.c -o main.o -l l2fwd -l dpdk -l numa -pthread -l dl

I guess the issue is in this step. DPDK uses link-level constructors (i.e. __attribute__((constructor))). See the definition of RTE_INIT() for example. Also there are callbacks etc. etc.

So to properly link with DPDK we have to:

• either use an rte.app.mk in the Makefile (see Development Kit Build System)

• or in case we need to use a custom build system, we need to link DPDK libraries after the --whole-archive option.