TensorFlow from sources problems

Question

I'm on a Fedora 26 distro

• kernel: 4.12.5
• gcc: 5.4 (5.3.1 is what is recommended, but I couldn't find it @ gnu's)
• bazel: 0.5.3 (which bazel outputs /usr/local/bin/bazel)
• PATH: ....:/usr/local/bin:.....
• CUDA 8.0

I git cloned from TensorFlow's repository, ran ./configure with the following (Kept only the essntials):

lease specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr]:/usr/local/cudnn
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: /home/elior/gcc54/bin/gcc

If anything more is needed let me know and I'll post it. Configuration seems to be finished by now and when I run

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

I get the following error

...... Cuda Configuration Error: Repository command failed find: ‘/usr/nvvm’: No such file or directory

Now few things that could have gone wrong that I could think of.

1. Using which nvcc outputs /usr/bin/nvcc but when the configuration asks me for the path for the CUDA compiler and I reply /usr/bin/nvcc it says that /usr/bin/nvcc/lib64/libcudart.so.8.0 could not be found, so I did a search and I found that file at /usr/lib64/libcudart.so.8.0 so that's why I've put the path as /usr
2. Pretty much the same thing with cudNN, I downloaded 5.1 from the site, and I've extracted it into /usr/local/cudnn but when I put in cudNN version I want to use as 5.1 it can't find /usr/local/cudnn/libcudnn.so.5.1 but I do have there a 5.0, so I just say "5" as the version and it works out

That's all I could come up by now... But I really want to get this installation done, any help would be appreciated.

Answer 1

we spend way too many days/hours on this GPU stuff. hopefully we can save you some time by sharing the following links:

AWS + Docker + CUDA + CuDNN + GPU + Spark + TensorFlow + JupyterHub

https://github.com/fluxcapacitor/pipeline/wiki/AWS-GPU-Tensorflow-Docker

Google + Docker + CUDA + CuDNN + GPU + Spark + TensorFlow + JupyterHub

https://github.com/fluxcapacitor/pipeline/wiki/GCP-GPU-TensorFlow-Docker

We use these instructions for meetups and conferences, etc. And we update them all the time when stuff breaks - which is quite often, unfortunately, with all the moving parts involved.

The Docker image we reference is here: https://github.com/fluxcapacitor/pipeline/blob/master/gpu.ml/Dockerfile.gpu

This Docker image extends from this: https://github.com/fluxcapacitor/pipeline/blob/master/package/gpu/cuda8/16.04/Dockerfile

which extends from this Nvidia Base Docker Image: FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04

This Nvidia Base Docker Image already includes the CuDNN libraries.

We have a need to build TensorFlow from source as we use a lot of TensorFlow's performance optimization utilities that must be built from source.

Hope that helps! More details and references available at the GitHub and DockerHub repos reference here: http://pipeline.ai

Answer 2

Sort of a workaround - use docker

nvidia-docker

tensorflow docker image

Answer 3

please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr

here you need to se the path to cuda installation directory; which is /usr/local/cuda (as it's also the default); Now you set it as /usr which is wrong; either you leave it as default or set it as /usr/local/cuda

Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr]:/usr/local/cudnn

here also the usual path that you need to set is /usr/local/cuda/

cudnn install

cp cudnn/lib64/cudnn* /usr/local/cuda/lib64
cp cudnn/include/* /usr/local/cuda/include