CODEWITHSUNDEEP
cinfinibandrdma
brooks94
brooks94

Reputation: 3936

"Local" RDMA for development

I'm trying to build and run the RDMA examples here. But because I'm just exploring, I don't have any hardware capable of managing RDMA. I get errors like this when I try to run the example code. 

librdmacm: couldn't read ABI version.
librdmacm: assuming: 4
CMA: unable to get RDMA device list
error: ec = rdma_create_event_channel() failed (returned zero/null).

Is there any "local" implementation of the RDMA functionality that I can use for development? I understand that the "R" in RDMA means remote, but I thought this might exist for testing/development purposes.

For reference, I'm trying this on an Ubuntu 14.04 box having installed the packages libibverbs-dev and librdmacm-dev in order to get the code to compile.

Upvotes: 11

Views: 2371

Answers (3)

user251384
user251384

Reputation: 184

Soft-RoCE and iWARP drivers have been integrated in the Linux kernel after this question was asked. Here is an example of how to configure these drivers on a system with an Ethernet interface that has name eth0:

# apt-get install -y iproute2
# modprobe rdma_rxe
# modprobe siw
# rdma link add eth0_rxe type rxe netdev eth0
# rdma link add eth0_siw type siw netdev eth0
# ls /sys/class/infiniband
eth0_rxe  eth0_siw

Upvotes: 2

Noich
Noich

Reputation: 15511

Perhaps this is what you were looking for:

https://community.mellanox.com/docs/DOC-2184

This is a Soft RoCE installation guide. Since it's thorough and long it's impossible to copy-paste here. Quoting:

Soft-RoCE is a software implementation of RoCE that allows RoCE to run on any Ethernet network adapter whether it offers hardware acceleration or not.

Soft-RoCE is released as part of upstream kernel 4.8 (or above). Mellanox OFED 4.0 or upstream driver could be used. If you install MLNX_OFED 4.0, you automatically get the Soft-RoCE kernel module and user space libraries.

This post demonstrates how to install and setup upstream Soft-RoCE (aka RXE), and is meant for IT managers and developers who wish to test RDMA on software over any 3rd party adapters.

Upvotes: -1

thenewguy617
thenewguy617

Reputation: 121

Have you tried using soft-iwarp?

http://voidreflections.blogspot.com/2011/03/how-to-install-soft-iwarp-on-ubuntu.html

https://gitorious.org/softiwarp

Upvotes: 2

Related Questions