TensorFlow on Windows: "Couldn't open CUDA library cudnn64_5.dll"

Question

Tensorflow just released windows support. I installed the gpu version and CUDA 8.0 and python 3.5. However, after I import the tensorflow I got the following error:

>>> import tensorflow
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cublas64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library cudnn64_5.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_dnn.cc:3459] Unable to load cuDNN DSO
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cufft64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library curand64_80.dll locally

Can someone help? Thanks!

Answer 1

It took me a lot as for Dan Bricklin.
I finally achieved it taking the same stpes but in order.

1. Removed the previous tensorflow from my conda environment.

2. Installed CUDA v9.1 for Windows 8 (standar installation with the default version of VisualStudio) and added "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1" to the path.

3. Downloaded and extracted cudnn-9.1-windows-7-x64 , moved it into "Program files" and added it to the path "C:\Program Files\cuda\bin".

4. Installed tensorflow with pip in my conda environment "pip install --ignore-installed --upgrade tensorflow".

Answer 2

And you could check your Environmental variable in this way:

import os
print("Environmental variable:", os.environ["PATH"])
C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;

My CUDA_path is "D:/CUDA/v8.0/bin", and I can not find my CUDA_path here. You will find your missing file here(such as "cublas64_80.dll", "cudnn64_5.dll", etc.) . The premise is that you had finish your CUDA installing.

If you can not find your CUDA_path in Environmental variable, you could add your CUDA_path manually: (The order of the following code is very important. Add the CUDA_path before import TensorFlow.)

import os
os.environ["PATH"] += ";D:/CUDA/v8.0/bin;" 
import tensorflow as tf

Or you could add your CUDA_path in CMD temporarily:

set PATH=%PATH%;"D:/CUDA/v8.0/bin"
python3 tensorflow_model.py

It work on my laptop(Windows10, Python3.6, Tensorflow-gpu==1.5). I think this two ways are simple solutions.

Answer 3

You can also put the cuda64_5.dll in the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin directory and it works fine.

You can use the folder GUI and load the .dll into the directory. Then you dont need to define the PATH every time you use Tensorflow.

Answer 4

As in my test, I found that TF 1.2 supports cudnn 64_5, and TF 1.3 supports cudnn 64_6. Hope this will help. :)

Answer 5

Tried pip3 install --upgrade tensorflow after tensorflow-gpu and it worked fine.

I think it's an issue only when trying pip3 install --upgrade tensorflow-gpu directly.

Answer 6

I just downloaded the cuda.dll file from this website:https://developer.nvidia.com/cudnn

and then moved the unzipped folder to where the rest of my anaconda libraries were.

I use pycharm, so it was easy to see where all of the external libraries were stored within anaconda. Hope this helps!

Answer 7

I had this problem and it took me several attempts to resolve it. This answer applies to Python 64 on Windows 64 I also have VS2017 installed with Python 3.6

From a clean Windows 64 machine Install Visual Studio 2015 (note: NOT vs2017 -- at least not yet). The community edition is free. Make sure you install the C++ compiler. You will need this to compile future python libs.

This will also clean up any problems with msvcp140.dll or msvcrt*.dlls. Alternatively you can install the VC Redistributable (but i recommend installing VS2015 instead as this will allow you to compile and install future python libraries).

Next, install VS2017 and this time also select Python and Data Learning (the scikit). This will default to installing Anaconda with Python 3.6 (built with VS2015). There are also some useful features

*Also make sure you have a compatible Nvidia card (see previous answers)

Then make sure you have the latest Nvidia drivers installed on your computer.

Then make sure you have installed the Nvidia libraries mentioned by Google and others including the Cuda bins.

*As of tensorflow 1.2, Python 3.6 is supported so the notes on creating a Python 3.5 environment are no longer necessary

There are a few problems i ran into with tensorflow 1.2. I also tried tensorflow 1.31rc2 in my environment

Problem #1 -- Firewalls (for those behind a firewall)

This will prevent installations via "pip install" To fix this, add --trusted-host pypi.python.org

For example

pip install tensorflow-gpu --trusted-host pypi.python.org

Problem #2 -- Upgrading to numpy 1.13.1

Installing tensorflow will upgrade to an incompatible version of numpy 1.13.1 (at least on my windows machine). To fix this, download the wheel at http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy look for numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl

Install this wheel before installing tensorflow. This will stop tensorflow from installing an incompatible version of the numpy 1.13.1 package

NOTE: see how to install wheels in other posts (pip install fullpath_of_wheel)

NOTE: if you have already installed tensorflow, make sure you force an upgrade using the --upgrade option with pip install)

Problem #3 -- Nvidia cuda packages in different locations

To test if you have the proper CU*.DLL packages use the "where" command (from a C:\ command prompt)

where cublas64_80.dll

where cudnn64_5.dll

where cufft64_80.dll

where nvcuda.dll

where curand64_80.dll

where cusolver64_80.dll

If your machine is like mine, you will find those dlls in multiple locations and sometimes they do not even exist. For instance, cublas64_80.dll was found in my c:\program files\Anaconda3 directory nvcuda.dll in my c:\system32\windows and so on. If you have matlab installed, it will have its own version. CNTK has its own versions also. Nvidia will put them in another directory. This is yet another problem. As mentioned by others, some of the DLLs you need are provided by Nvidia in a zip file.

Instead of trying to fix up your paths, I recommend trying this first instead

Collect the 6 dlls mentioned above and place them in ONE directory such as c:\tfexperiment

Then cd into c:\tfexperiment

run python.exe from this location. Windows will now look for the dlls in the current path first

now once python loads type in

import tensorflow as tf

it should work for you (hopefully). This was the only way I was able to get it to work on my machine. If you get this far, you can simply add c:\tfexperiment as your first path in the path environment variable. Or you can figure out the correct path order.

If it STILL does not work, you can take it one step further by downloading procmon.exe from Microsoft. Run procmon.exe. Filter on the executable python.exe (sorry i dont have time to explain how to use procmon). Now go back to your python prompt and type "import tensorflow as tf" again. procmon should have many lines of information. You may want to filter on loadimage. This will tell you what dlls it is loading. Note that .pyd extensions are also DLLs. The last .dll that loaded (or failed to load) is probably the one that caused problems.

Answer 8

Also make sure that there is no additional cudnn64_5.dll (of the wrong version) sitting in another path.

So if you put the DLL in ...\CUDA\v8.0\bin in the past, remember that you did and do not put a newer version into ...\CuDNN\v5.1\bin without removing the old one from the other directory.

Answer 9

For those of you that land here because of:

I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library cublas64_80.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_blas.cc:2294] Unable to load cuBLAS DSO.
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cudnn64_5.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library cufft64_80.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_fft.cc:344] Unable to load cuFFT DSO.
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library curand64_80.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_rng.cc:338] Unable to load cuRAND DSO.

You need to add the CUDA regular path. I don't know why they didn't just put them together as one download. Very silly.

C:\Users\user>set PATH=%PATH%;C:\tools\cuda\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin

Answer 10

In addition to the answers above make sure that you've downloaded the supported version of cuDNN. Currently TensorFlow supports the older cuDNN v.5.1 while there is a newer cuDNN 6.0 available on Nvidia site. I had such errors with 6.0. When I rolled back to 5.1 everything worked.

• Check TensorFlow requirments here: https://www.tensorflow.org/install/install_windows
• Download the supported version of cuDNN from here: https://developer.nvidia.com/rdp/cudnn-download

Answer 11

TL;DR: To use cuDNN with TensorFlow, the file cudnn64_5.dll must be in a directory that is in your %PATH% environment variable. Note that cuDNN is a separate download from CUDA, and you must download version 5.1 of cuDNN.

First of all, note that cuDNN is not distributed with the rest of the CUDA toolkit, so you will need to download it separately from the NVIDIA website. On Windows, it is distributed as a ZIP archive, so you must extract it and find the directory containing cudnn64_5.dll. For example, if you extract it to C:\tools\cuda, the DLL will be in C:\tools\cuda\bin\cudnn64_5.dll. Finally, you can add it to your path by typing the following at the command prompt:

C:\> set PATH=%PATH%;C:\tools\cuda\bin
C:\> python
...
>>> import tensorflow as tf

Answer 12

You have to download cudnn of your system, and extract it in the CUDA_PATH.

My CUDA_PATH is C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0