VirtualEnv on Windows, pip fails to install packages with C-extensions

Question

I'm trying to modernize a little, but for whatever reason pip fails to install anything with a C-extension. The nasty offenders being MySQLdb, PIL, Fabric. I can install them with exe's, but modern requirements techniques won't work.

For example, here's what PIL does (virtualtesting is the environment, and has nothing else in it yet):

(virtualtesting) C:\Users\jgoldstein\Localhost\virtualtesting>pip install pil
Downloading/unpacking pil
  Downloading PIL-1.1.7.tar.gz (506Kb): 506Kb downloaded
  Running setup.py egg_info for package pil
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py

Installing collected packages: pil
  Running setup.py install for pil
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py
    building '_imaging' extension
    error: Unable to find vcvarsall.bat
    Complete output from command C:\Users\jgoldstein\Localhost\virtualtesting\Scripts\python.exe -c "import setuptools;__file__='C:\\Users\\jgoldstein\\Localhost\\virtualtesting\\build\\pil\\setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, 'exec'))" install --single-version-externally-managed --record c:\users\jgolds~1\appdata\local\temp\pip-wbez3q-record\install-record.txt --install-headers C:\Users\jgoldstein\Localhost\virtualtesting\include\site\python2.7:
    WARNING: '' not a valid package name; please use only.-separated package names in setup.py

running install

running build

running build_py

creating build

creating build\lib.win32-2.7

copying PIL\ArgImagePlugin.py -> build\lib.win32-2.7

copying PIL\BdfFontFile.py -> build\lib.win32-2.7

copying PIL\BmpImagePlugin.py -> build\lib.win32-2.7

copying PIL\BufrStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\ContainerIO.py -> build\lib.win32-2.7

copying PIL\CurImagePlugin.py -> build\lib.win32-2.7

copying PIL\DcxImagePlugin.py -> build\lib.win32-2.7

copying PIL\EpsImagePlugin.py -> build\lib.win32-2.7

copying PIL\ExifTags.py -> build\lib.win32-2.7

copying PIL\FitsStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\FliImagePlugin.py -> build\lib.win32-2.7

copying PIL\FontFile.py -> build\lib.win32-2.7

copying PIL\FpxImagePlugin.py -> build\lib.win32-2.7

copying PIL\GbrImagePlugin.py -> build\lib.win32-2.7

copying PIL\GdImageFile.py -> build\lib.win32-2.7

copying PIL\GifImagePlugin.py -> build\lib.win32-2.7

copying PIL\GimpGradientFile.py -> build\lib.win32-2.7

copying PIL\GimpPaletteFile.py -> build\lib.win32-2.7

copying PIL\GribStubImagePlugin.py -> build\lib.win32-2.7

copying PIL\Hdf5StubImagePlugin.py -> build\lib.win32-2.7

copying PIL\IcnsImagePlugin.py -> build\lib.win32-2.7

copying PIL\IcoImagePlugin.py -> build\lib.win32-2.7

copying PIL\Image.py -> build\lib.win32-2.7

copying PIL\ImageChops.py -> build\lib.win32-2.7

copying PIL\ImageCms.py -> build\lib.win32-2.7

copying PIL\ImageColor.py -> build\lib.win32-2.7

copying PIL\ImageDraw.py -> build\lib.win32-2.7

copying PIL\ImageDraw2.py -> build\lib.win32-2.7

copying PIL\ImageEnhance.py -> build\lib.win32-2.7

copying PIL\ImageFile.py -> build\lib.win32-2.7

copying PIL\ImageFileIO.py -> build\lib.win32-2.7

copying PIL\ImageFilter.py -> build\lib.win32-2.7

copying PIL\ImageFont.py -> build\lib.win32-2.7

copying PIL\ImageGL.py -> build\lib.win32-2.7

copying PIL\ImageGrab.py -> build\lib.win32-2.7

copying PIL\ImageMath.py -> build\lib.win32-2.7

copying PIL\ImageMode.py -> build\lib.win32-2.7

copying PIL\ImageOps.py -> build\lib.win32-2.7

copying PIL\ImagePalette.py -> build\lib.win32-2.7

copying PIL\ImagePath.py -> build\lib.win32-2.7

copying PIL\ImageQt.py -> build\lib.win32-2.7

copying PIL\ImageSequence.py -> build\lib.win32-2.7

copying PIL\ImageShow.py -> build\lib.win32-2.7

copying PIL\ImageStat.py -> build\lib.win32-2.7

copying PIL\ImageTk.py -> build\lib.win32-2.7

copying PIL\ImageTransform.py -> build\lib.win32-2.7

copying PIL\ImageWin.py -> build\lib.win32-2.7

copying PIL\ImImagePlugin.py -> build\lib.win32-2.7

copying PIL\ImtImagePlugin.py -> build\lib.win32-2.7

copying PIL\IptcImagePlugin.py -> build\lib.win32-2.7

copying PIL\JpegImagePlugin.py -> build\lib.win32-2.7

copying PIL\McIdasImagePlugin.py -> build\lib.win32-2.7

copying PIL\MicImagePlugin.py -> build\lib.win32-2.7

copying PIL\MpegImagePlugin.py -> build\lib.win32-2.7

copying PIL\MspImagePlugin.py -> build\lib.win32-2.7

copying PIL\OleFileIO.py -> build\lib.win32-2.7

copying PIL\PaletteFile.py -> build\lib.win32-2.7

copying PIL\PalmImagePlugin.py -> build\lib.win32-2.7

copying PIL\PcdImagePlugin.py -> build\lib.win32-2.7

copying PIL\PcfFontFile.py -> build\lib.win32-2.7

copying PIL\PcxImagePlugin.py -> build\lib.win32-2.7

copying PIL\PdfImagePlugin.py -> build\lib.win32-2.7

copying PIL\PixarImagePlugin.py -> build\lib.win32-2.7

copying PIL\PngImagePlugin.py -> build\lib.win32-2.7

copying PIL\PpmImagePlugin.py -> build\lib.win32-2.7

copying PIL\PsdImagePlugin.py -> build\lib.win32-2.7

copying PIL\PSDraw.py -> build\lib.win32-2.7

copying PIL\SgiImagePlugin.py -> build\lib.win32-2.7

copying PIL\SpiderImagePlugin.py -> build\lib.win32-2.7

copying PIL\SunImagePlugin.py -> build\lib.win32-2.7

copying PIL\TarIO.py -> build\lib.win32-2.7

copying PIL\TgaImagePlugin.py -> build\lib.win32-2.7

copying PIL\TiffImagePlugin.py -> build\lib.win32-2.7

copying PIL\TiffTags.py -> build\lib.win32-2.7

copying PIL\WalImageFile.py -> build\lib.win32-2.7

copying PIL\WmfImagePlugin.py -> build\lib.win32-2.7

copying PIL\XbmImagePlugin.py -> build\lib.win32-2.7

copying PIL\XpmImagePlugin.py -> build\lib.win32-2.7

copying PIL\XVThumbImagePlugin.py -> build\lib.win32-2.7

copying PIL\__init__.py -> build\lib.win32-2.7

running build_ext

building '_imaging' extension

error: Unable to find vcvarsall.bat

----------------------------------------
Command C:\Users\jgoldstein\Localhost\virtualtesting\Scripts\python.exe -c "import setuptools;__file__='C:\\Users\\jgoldstein\\Localhost\\virtualtesting\\build\\pil\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record c:\users\jgolds~1\appdata\local\temp\pip-wbez3q-record\install-record.txt --install-headers C:\Users\jgoldstein\Localhost\virtualtesting\include\site\python2.7 failed with error code 1
Storing complete log in C:\Users\jgoldstein\AppData\Roaming\pip\pip.log

It's Python 2.7 (32 bit), VirtualEnv 1.7, Windows 7 (64 bit). And I'm... dry.

So the first question: can this be fixed?

The second question: it seems if I run exe installers to the main site-packages and copy the results into the VirtualEnv's directory, it works okay. How bad of an idea is it to do this if I want to go to a pip+requirements.txt model of doing things?

(Obviously, on the server all this stuff works nicely.)

Answer 1

I can't answer the second question but I have a hint to simplify installing a package to the virtual environment:

You can activate your virtualenv and then run easy_install path_to_exe_installer and the package gets installed directly into the virtualenv.

Answer 2

Turns out this was a bug is distutils. Still no known solution if you use GCC.

Answer 3

For the first question, have look at: error: Unable to find vcvarsall.bat

Can't answer to the second one, i don't know about it :|