Does Tensorflow support Tesla K80

Question

I'm running Tensorflow 0.8 on Tesla K80 with CUDA 7.5 and CUDNN v5. Everything is fine but two devices can't access each other.

The warning log is listed below. Thank you.

I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 0 to device ordinal 2
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 0 to device ordinal 3
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 1 to device ordinal 2
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 1 to device ordinal 3
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 2 to device ordinal 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 2 to device ordinal 1
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 3 to device ordinal 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:59] cannot enable peer access from device ordinal 3 to device ordinal 1

Answer 1

I bet that you have some multi-socket configuration like this one:

were each K80 is not sharing the same PCIe root complex. Then, peer-to-peer accesses from GPU0 to GPU1 are allowed, but from GPU0 to GPU2/GPU3 are not.

Tensorflow should be able to detect this kind of system and perform manual copies between GPUs.

Answer 2

Yes, TensorFlow should work on a Tesla K80 GPU (although it is not one of the officially tested devices). As Robert Crovella points out in his comment, those warning messages merely state that your system isn't configured for peer-to-peer access between the various GPU devices, but you should still be able to run TensorFlow on them.