CODEWITHSUNDEEP
cuda
superscalar
superscalar

Reputation: 725

CUDA driver version is insufficient for CUDA runtime version

I got the message:

"cutilCheckMsg() CUTIL CUDA error :
kernel launch failure : CUDA driver
version is insufficient for CUDA
runtime version."

While trying to run an example source code. Also happens for the function cutilSafeCall.

I am using:

Upvotes: 43

Views: 185340

Answers (12)

nikg
nikg

Reputation: 1

I had to add -Wno-deprecated-gpu-targets as a flag during compilation.

Upvotes: 0

lenin
lenin

Reputation: 1011

This problem can also be because of incorrect environment setup, e.g. Docker image setup. Although the driver itself is correct, sufficient for your program. If your LD_LIBRARY_PATH points to the wrong driver, it can throw this error. In my case, i get this error when using /usr/local/nvidia/lib/libcuda.so, and if I use /usr/local/nvidia/lib64/libcuda.so everything goes right.

Upvotes: 0

Kees Schollaart
Kees Schollaart

Reputation: 836

In my case, I had to run my docker container with nvidia-docker run ... instead of docker run ...

Upvotes: 3

alexei
alexei

Reputation: 2311

Counterintuitively, this error also happens if libcuda.so is not found, even when versions reported by nvidia-smi match perfectly. This library is part of nvidia-drivers package (On CentOS: nvidia-driver-latest-cuda-libs, on Gentoo x11-drivers/nvidia-drivers). It is possible to have the CUDA Tookit with nvcc and libcudart installed and building your app fine, but the drivers part not installed, causing this error.

To diagnose whether this is the reason, use strace:

strace -f -e trace=file ./your_cuda_app

and check for open calls to libcuda.so*, at least one of them should return with a success code, like so:

4928  open("/lib64/libcuda.so.1", O_RDONLY|O_CLOEXEC) = 3

Upvotes: 10

shalini singh
shalini singh

Reputation: 1

This is what has worked for me :

  1. Installing TensorFlow version 1.14, installing keras
  2. Install CUDA 10.0 from https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exenetwork for windows
  3. Download the compatible CuDNN i.e 7.4.2 from https://developer.nvidia.com/rdp/cudnn-archive
  4. set the following in the path variable cudnn-10.0-windows10-x64-v7.4.2.24\cuda\bin cudnn-10.0-windows10-x64-v7.4.2.24\cuda\include cudnn-10.0-windows10-x64-v7.4.2.24\cuda\lib\x64
  5. Restart the Pc

Upvotes: 0

Frank
Frank

Reputation: 144

I saw the same at runtime with the latest driver on Mac OS 10.6. 

cudaError_t error = cudaGetDevice(&device);
printf("%s\n", cudaGetErrorString(error));

I went back to the developer site, downloaded the driver again and now it runs. http://developer.nvidia.com/object/cuda_3_1_downloads.html#MacOS

Upvotes: 12

user3112759
user3112759

Reputation: 9

I also had similar problem, updated my graphic driver but the problem still remained. I finally decided to remove Cuda 9.2 and install Cuda 8, it solved my issue.

Upvotes: 0

Fabiano Tarlao
Fabiano Tarlao

Reputation: 3232

My cent,

with Linux/Unix this error may be related to the selected GPU mode (Performance/Power Saving Mode), when you select (with nvidia-settings utiliy) the integrated Intel GPU and you execute the deviceQuery script... you get this error:

-> CUDA driver version is insufficient for CUDA runtime version

But this error is misleading, by selecting back the NVIDIA(Performance mode) with nvidia-settings utility the problem disappears.

It is not a version problem.

Regards

P.s: "Power Saving Mode" tells Optimus to activate the CPU integrated Intel GPU

Upvotes: 5

herrera
herrera

Reputation: 117

Maybe it is related to the TBB lib: Error OpenCV with CUDA using TBB for multiple GPUs

Try rebuilding it making sure you passed the following parameters to CMake (assuming you already installed "tbb" and "tbb-devel" packages:

-D WITH_TBB=YES -D TBB_INCLUDE_DIRS=/usr/include/tbb

Upvotes: -1

Dongwei Wang
Dongwei Wang

Reputation: 495

CUDA driver version is insufficient for CUDA runtime version: means your GPU can`t been manipulated by the CUDA runtime API, so you need to update your driver.

Upvotes: 3

Meghana
Meghana

Reputation: 51

You can either download the latest driver OR use an older toolkit version to compile your code.

Upvotes: 5

Tom
Tom

Reputation: 21138

You need to ensure that your driver version matches or exceeds your CUDA Toolkit version.

For 2.3 you need a 190.x driver, for 3.0 you need 195.x and for 3.1 you need 256.x (actually anything up to the next multiple of five is ok, e.g. 258.x for 3.1).

You can check your driver version by either running the deviceQueryDrv SDK sample or go into the NVIDIA Control Panel and choose System Information.

Download an updated driver from www.nvidia.com/drivers.

Upvotes: 24

Related Questions