Error to load compiler libraries while importing Theano: fatal error C1083

Question

I'm trying to use Theano with my GPU for several days in my Python environment. While importing theano, I obtain this error:

d:\anaconda2\include\pyconfig.h(239) : fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory

I tried to include "D:\Anaconda2\MinGW\x86_64-w64-mingw32\include" (this folder contains "pyconfig.h") in PYTHONPATH, in PATH and in .theanorc.txt, with the same error message.

Do you have any ideas to connect anaconda/cuda/nvcc to the compiler in Theano?

Here is my configuration:

• OS: Windows 7
• GPU: GeForce GTX 950
• cl.exe : D:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin
• nvcc.exe : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
• Python IDE: Spyder from Anaconda 2, in Python 2.
• Theano 0.8.2

I can "import theano" with the CPU, but I want to use GPU for computations.

After typing "import theano" (with device=gpu), I can see a console window launching "nvcc.exe" before showing the error.

I tested CUDA with Visual Studio 2012 and it's working, for example "bilateralFilter" works in Visual Studio without error:

Found 1 CUDA Capable device(s) supporting CUDA
Device 0: "GeForce GTX 950"
CUDA Runtime Version: 8.0
CUDA Compute Capability: 5.2
...
Running Standard Demonstration with GLUT loop...

• Here is my theanorc.txt file (the same error occurs with or without [nvcc] and [cuda])

  [global]
  device = gpu
  floatX = float32
  
  [cuda]
  root = -LC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
  
  [nvcc]
  flags = -LD:\Anaconda\libs
  fastmath = True
  compiler-bindir = -LD:\Anaconda2\MinGW\x86_64-w64-mingw32\include
  

• Here is my PATH

  D:\Anaconda2\MinGW\x86_64-w64-mingw32\include
  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp
  C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin
  D:\Anaconda2
  D:\Anaconda2\Scripts
  D:\Anaconda2\Library\bin
  C:\Program Files (x86)\Windows Kits\8.0\Windows Performance Toolkit\
  C:\Program Files\Microsoft SQL Server\110\Tools\Binn\
  

• Here is my PYTHONPATH

  D:\Anaconda2\MinGW\x86_64-w64-mingw32\include
  C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC
  C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\amd64
  C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin
  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
  

• Here is the complete error message after typing "import theano" in Spyder (with device=gpu in theanorc.txt):

  1 #define _CUDA_NDARRAY_C
  2 
  3 #include <Python.h>
  4 #include <structmember.h>
  5 #include "theano_mod_helper.h"
  6 
  7 #include <numpy/arrayobject.h>
  8 #include <iostream>
  ...
  5358 /*
  5359   Local Variables:
  5360   mode:c++
  5361   c-basic-offset:4
  5362   c-file-style:"stroustrup"
  5363   indent-tabs-mode:nil
  5364   fill-column:79
  5365   End:
  5366 */
  5367 // vim: filetype=cpp:expandtab:shiftwidth=4:tabstop=8:softtabstop=4:textwidth=79 :
  5368 
  ===============================
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(849) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(1787) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(2637) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(3492) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(4431) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(5345) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(6252) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(7142) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\math_functions.h(7956) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\device_functions.h : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\device_functions.h(774) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\device_functions.h(1618) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\device_double_functions.h : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\sm_20_intrinsics.h : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\sm_20_intrinsics.h(943) : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss
  d:\anaconda2\include\pyconfig.h(239) : fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
  ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 2, 'for cmd', 'nvcc -shared -O3 -LD:\\Anaconda\\libs -use_fast_math -Xlinker /DEBUG -D HAVE_ROUND -m64 -Xcompiler -DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD -ID:\\Anaconda2\\lib\\site-packages\\theano\\sandbox\\cuda -ID:\\Anaconda2\\lib\\site-packages\\numpy\\core\\include -ID:\\Anaconda2\\include -ID:\\Anaconda2\\lib\\site-packages\\theano\\gof -o C:\\Users\\Administrator\\AppData\\Local\\Theano\\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_60_Stepping_3_GenuineIntel-2.7.13-64\\cuda_ndarray\\cuda_ndarray.pyd mod.cu -LD:\\Anaconda2\\libs -LD:\\Anaconda2 -lcublas -lpython27 -lcudart')
  WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: cuda unavailable)
  nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
  mod.cu
  
  ['nvcc', '-shared', '-O3', '-LD:\\Anaconda\\libs', '-use_fast_math', '-Xlinker', '/DEBUG', '-D HAVE_ROUND', '-m64', '-Xcompiler', '-DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,/Zi,/MD', '-ID:\\Anaconda2\\lib\\site-packages\\theano\\sandbox\\cuda', '-ID:\\Anaconda2\\lib\\site-packages\\numpy\\core\\include', '-ID:\\Anaconda2\\include', '-ID:\\Anaconda2\\lib\\site-packages\\theano\\gof', '-o', 'C:\\Users\\Administrator\\AppData\\Local\\Theano\\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_60_Stepping_3_GenuineIntel-2.7.13-64\\cuda_ndarray\\cuda_ndarray.pyd', 'mod.cu', '-LD:\\Anaconda2\\libs', '-LD:\\Anaconda2', '-lcublas', '-lpython27', '-lcudart']
  

Answer 1

I found a solution for my problem.

Short answer

1. Add C:\Windows\System32 to PATH
2. Launch C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\amd64\vcvars64.bat

End.

For information, here is my new current PATH:

C:\Windows\System32
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\amd64
D:\Anaconda2
D:\Anaconda2\Scripts
D:\Anaconda2\Library\bin

Here is my PYTHONPATH:

nothing

Here is my .theanorc.txt:

[global]
device=gpu
floatx = float32

Some details

In addition with PATH, one needs to configure Windows registry, as explained in this post: compilation error in visual studio linked with python26 . It is done by launching vcvars64.bat

I did this before, but I forgot to notice that a message appeared:

ERROR: Cannot determine the location of the VS Common Tools folder.

According to https://social.msdn.microsoft.com/Forums/en-US/78703f6b-f610-456c-b770-76a12be3e1ae/error-cannot-determine-the-location-of-the-vs-common-tools-folder?forum=vssetup , the solution is to add C:\Windows\System32 to PATH.