Cuda GPUDirect to NIC/Harddrive?

Question

I am currently writing a CUDA application and am running into a few IO issues "feeding the beast."

I am wondering if there is any way that I can directly read data from a RAID controller or NIC and have that data sent directly to the GPU. What I'm trying to accomplish is shown directly on slide #3 of the following presentation: http://developer.download.nvidia.com/devzone/devcenter/cuda/docs/GPUDirect_Technology_Overview.pdf.

That being said, apparently this has been answered already here: Is it possible to access hard disk directly from gpu?, however the presentation that I've attached leads to believe all I need is to set an environment variable in Linux (but it doesn't offer any useful code snippets/examples).

Therefore, I'm wondering if it is possible to read data directly from a NIC/RAID controller into the GPU and what would be required to do so? Would I need to write my own driver for the hardware? Are there any examples where certain copies are avoided?

Thanks in advance for the help.

Answer 1

GPUDirect is a technology "umbrella term", which in general is a brand referring to technologies that enable direct data transfer to and/or from a GPU, somehow bypassing unnecessary trips through host memory.

GPUDirect v1 is a technology that works with specific infiniband adapters, and enables the sharing of a data buffer between the GPU driver and the IB driver. This technology has mostly been superceded by GPUDirect (v3) RDMA. This v1 technology does not enable general usage with any NIC. The environment variable reference:

however the presentation that I've attached leads to believe all I need is to set an environment variable in Linux

refers to enabling GPUDirect v1. It is not a general purpose NIC enabler.

GPUDirect v2 is also called GPUDirect Peer-to-Peer, and it is for transfer of data between two CUDA GPUs on the same PCIE fabric only. It does not enable interoperability with any other kind of device.

GPUDirect v3 is also called GPUDirect RDMA.

Therefore, I'm wondering if it is possible to read data directly from a NIC/RAID controller into the GPU and what would be required to do so?

Today, the canonical use case for GPUDirect RDMA is with a Mellanox Infiniband (IB) adapter. (It can also be made to work, perhaps with assistance from Mellanox, using a Mellanox Ethernet Adapter and RoCE). If this fits your definition of "NIC", then it's possible by loading a proper software stack, assuming you have appropriate hardware. The GPU and the IB device need to be on the same PCIE fabric, which means they need to be attached to the same PCIE root complex (effectively, connected to the same CPU socket). When used with a Mellanox IB adapter, typical usage would involve a GPUDirect RDMA-aware MPI.

If you have your own unspecified NIC or RAID controller, and you don't already have a GPUDirect RDMA linux device driver for it, then it's not possible to use GPUDirect. (If there is a GPUDirect RDMA driver for it, contact the manufacturer or driver provider for assistance.) If you have access to the driver source code, and are familiar with writing your own linux device drivers, you could try crafting your own GPUDirect driver. The steps involved are beyond the scope of my answer, but the starting point is documented here.

Would I need to write my own driver for the hardware?

Yes, if you don't already have a GPUDirect RDMA driver for it, one would need to be written.

Are there any examples where certain copies are avoided?

The GPUDirect RDMA MPI link gives examples and explains how GPUDirect RDMA can avoid unnecessary device<->host data copies during the transfer of data from GPU to IB adapter. In general, data can be transferred directly (over PCIE) from memory on the GPU device to memory on the IB device (or vice-versa) with no trip through host memory (GPUDirect v1 did not achieve this.)

UPDATE: NVIDIA has recently announced a new GPU Direct technology called GPU Direct Storage.