CODEWITHSUNDEEP
pythoncvisual-c++lzma
user111086
user111086

Reputation:

Installing/compiling pylzma (lzma python binding)

I've already posted this question on the authors website, but I thought I might ask here as well.

I've been trying to install pylzma with this setup:

With a simple easy_install pylzma I got this:

cl : Command line warning D9025 : overriding '/MD' with '/MT' pylzma.c src/pylzma/pylzma.c(85) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

src/pylzma/pylzma.c(102) : error C2275: 'CSha256' : illegal use of this type as an expression c:\users\xavier.lapointe\appdata\local\temp\easy_install-2mfkqu\pylzma-0.4.3\src\pylzma../7zip/C/Sha256.h(18) : see declaration of 'CSha256'

src/pylzma/pylzma.c(102) : error C2146: syntax error : missing ';' before identifier 'sha'

src/pylzma/pylzma.c(102) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(103) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(103) : warning C4133: 'function' : incompatible types

  • from 'int *' to 'CSha256 *'

src/pylzma/pylzma.c(104) : error C2143: syntax error : missing ';' before 'type'

src/pylzma/pylzma.c(105) : error C2143: syntax error : missing ';' before 'type'

src/pylzma/pylzma.c(106) : error C2143: syntax error : missing ';' before 'type'

src/pylzma/pylzma.c(107) : error C2143: syntax error : missing ';' before 'type'

src/pylzma/pylzma.c(108) : error C2065: 'round' : undeclared identifier

src/pylzma/pylzma.c(108) : error C2065: 'round' : undeclared identifier

src/pylzma/pylzma.c(108) : error C2065: 'rounds' : undeclared identifier

src/pylzma/pylzma.c(108) : error C2065: 'round' : undeclared identifier

src/pylzma/pylzma.c(109) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(109) : warning C4133: 'function' : incompatible types

  • from 'int *' to 'CSha256 *'

src/pylzma/pylzma.c(110) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(110) : warning C4133: 'function' : incompatible types

  • from 'int *' to 'CSha256 *'

src/pylzma/pylzma.c(111) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(111) : warning C4133: 'function' : incompatible types

  • from 'int *' to 'CSha256 *'

src/pylzma/pylzma.c(111) : error C2065: 'temp' : undeclared identifier

src/pylzma/pylzma.c(112) : error C2065: 'i' : undeclared identifier

src/pylzma/pylzma.c(112) : error C2065: 'i' : undeclared identifier

src/pylzma/pylzma.c(112) : error C2065: 'i' : undeclared identifier

src/pylzma/pylzma.c(113) : error C2065: 'temp' : undeclared identifier

src/pylzma/pylzma.c(113) : error C2065: 'i' : undeclared identifier

src/pylzma/pylzma.c(113) : error C2109: subscript requires array or pointer type

src/pylzma/pylzma.c(116) : error C2065: 'sha' : undeclared identifier

src/pylzma/pylzma.c(116) : warning C4133: 'function' : incompatible types

  • from 'int *' to 'CSha256 *'

error: Setup script exited with error: command '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe"' failed with exit status 2

Then after googling I figured it was about the compiler (c90 vs c99), some people suggested to change the .c extension to .cpp. So I did this and changed the setup.py accordingly (mostly files name).

Then success, it compiles after a few tweaks in CpuArch.c(pp) and another file which had syntax errors.

The thing is now the linker choke:

C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\dev\pylzma\build\temp.win-amd64-2.6\Release\src\pylzma /LIBPATH:C:\Python26\libs /LIBPATH:C:\Python26\PCbuild\amd64 user32.lib oleaut32.lib /EXPORT:initpylzma build\temp.win-amd64-2.6\Release\src/pylzma/pylzma.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_decompressobj.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_compressfile.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_decompress.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_compress.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_streams.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_aes.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_decompress_compat.obj build\temp.win-amd64-2.6\Release\src/pylzma/pylzma_decompressobj_compat.obj build\temp.win-amd64-2.6\Release\src\sdk\LzFind.obj build\temp.win-amd64-2.6\Release\src\sdk\LzmaDec.obj build\temp.win-amd64-2.6\Release\src\sdk\LzmaEnc.obj build\temp.win-amd64-2.6\Release\src\7zip\C\CpuArch.obj build\temp.win-amd64-2.6\Release\src\7zip\C\Aes.obj build\temp.win-amd64-2.6\Release\src\7zip\C\AesOpt.obj build\temp.win-amd64-2.6\Release\src\7zip\C\Sha256.obj build\temp.win-amd64-2.6\Release\src\compat\LzmaCompatDecode.obj build\temp.win-amd64-2.6\Release\src/sdk/LzFindMt.obj build\temp.win-amd64-2.6\Release\src/sdk/Threads.obj /OUT:build\lib.win-amd64-2.6\pylzma.pyd /IMPLIB:build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.lib /MANIFESTFILE:build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.pyd.manifest

pylzma.obj : warning LNK4197: export 'initpylzma' specified multiple times; using first specification Creating library build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.lib and object build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.exp

pylzma.obj : error LNK2001: unresolved external symbol "char const * const doc_decompress" (?doc_decompress@@3QBDB)

pylzma.obj : error LNK2001: unresolved external symbol "struct _object * __cdecl pylzma_decompress(struct _object *,struct _object *)" (?pylzma_decompress@@YAPEAU_object@@PEAU1@0@Z)

pylzma.obj : error LNK2001: unresolved external symbol "char const * const doc_compress" (?doc_compress@@3QBDB)

Aes.obj : error LNK2019: unresolved external symbol "void __cdecl AesCtr_Code_Intel(unsigned int *,unsigned char *,unsigned __int64)" (?AesCtr_Code_Intel@@YAXPEAIPEAE_K@Z) referenced in function AesGenTables

Aes.obj : error LNK2019: unresolved external symbol "void __cdecl AesCbc_Decode_Intel(unsigned int *,unsigned char *,unsigned __int64)" (?AesCbc_Decode_Intel@@YAXPEAIPEAE_K@Z) referenced in function AesGenTables

Aes.obj : error LNK2019: unresolved external symbol "void __cdecl AesCbc_Encode_Intel(unsigned int *,unsigned char *,unsigned __int64)" (?AesCbc_Encode_Intel@@YAXPEAIPEAE_K@Z) referenced in function AesGenTables

AesOpt.obj : error LNK2019: unresolved external symbol _mm_aesenclast_si128 referenced in function "void __cdecl AesCbc_Encode_Intel(union __m128i *,union __m128i *,unsigned __int64)" (?AesCbc_Encode_Intel@@YAXPEAT__m128i@@0_K@Z)

AesOpt.obj : error LNK2019: unresolved external symbol _mm_aesenc_si128 referenced in function "void __cdecl AesCbc_Encode_Intel(union __m128i *,union __m128i *,unsigned __int64)" (?AesCbc_Encode_Intel@@YAXPEAT__m128i@@0_K@Z)

AesOpt.obj : error LNK2019: unresolved external symbol _mm_aesdeclast_si128 referenced in function "void __cdecl AesCbc_Decode_Intel(union __m128i *,union __m128i *,unsigned __int64)" (?AesCbc_Decode_Intel@@YAXPEAT__m128i@@0_K@Z)

AesOpt.obj : error LNK2019: unresolved external symbol _mm_aesdec_si128 referenced in function "void __cdecl AesCbc_Decode_Intel(union __m128i *,union __m128i *,unsigned __int64)" (?AesCbc_Decode_Intel@@YAXPEAT__m128i@@0_K@Z) build\lib.win-amd64-2.6\pylzma.pyd : fatal error LNK1120: 10 unresolved externals

error: command '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe"' failed with exit status 1120

Now I've a hard time trying to figure out what these linker issues are, even with the msdn doc (about linkers error). Probably due to my lack of knowledge with C/C++.

So basically my question(s) are:

EDIT Haven't got much answer, so I think I'll add some questions:

I'm looking more for hints that might help than a fully working answer.

Upvotes: 2

Views: 5288

Answers (1)

Zart
Zart

Reputation: 1441

I have installed pylzma to try this out myself. For python 2.6 it builds fine with mingw, but msvc compiler chokes for python2.4-2.7. Let's see errors first:

src/pylzma/pylzma.c(102) : error C2275: 'CSha256' : illegal use of this type as an expression c:\users\xavier.lapointe\appdata\local\temp\easy_install-2mfkqu\pylzma-0.4.3\src\pylzma../7zip/C/Sha256.h(18) : see declaration of 'CSha256'

Code in question is:

} else {
    Py_BEGIN_ALLOW_THREADS
    Sha256_Init(&sha);
    CSha256 sha;
    long round;
    int i;
    long rounds = (long) 1 << cycles;
    unsigned char temp[8] = { 0,0,0,0,0,0,0,0 };
    for (round = 0; round < rounds; round++) {

which is clearly invalid C since it's not allowed to declare variables after code. When you renamed .c to .cpp you avoided this error since C++ allows for this (and mingw supports this for C apparently). But switching to C++ led to name mangling and linking errors. To fix those you could place extern "C" { ... } around all the code.

It's better to fix src/pylzma/pylzma.c htough and the fix is trivial - move Py_BEGIN_ALLOW_THREADS and Sha256_Init(&sha); after variable declarations:

} else {
    CSha256 sha;
    long round;
    int i;
    long rounds = (long) 1 << cycles;
    unsigned char temp[8] = { 0,0,0,0,0,0,0,0 };
    Py_BEGIN_ALLOW_THREADS
    Sha256_Init(&sha);
    for (round = 0; round < rounds; round++) {

Now pylzma compiles fine but fails to run manifest tool after linking:

Creating library build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.lib and o bject build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.exp C:\Program Files\Microsoft SDKs\Windows\v7.0\bin\x64\mt.exe -nologo -manifest bu ild\temp.win-amd64-2.6\Release\src/pylzma\pylzma.pyd.manifest -outputresource:bu ild\lib.win-amd64-2.6\pylzma.pyd;2

build\temp.win-amd64-2.6\Release\src/pylzma\pylzma.pyd.manifest : general error c1010070: Failed to load and parse the manifest. The system cannot find the file specified. error: command 'mt.exe' failed with exit status 31

Looking in build directory reveals that there is no pylzma.pyd.manifest there though it's seen from output that link.exe has /MANIFEST:... switch. Quick googling for "link didn't create manifest" finds http://bugs.python.org/issue4431 where it's explained that when using /MT switch manifest doesn't get created and the solution is to add /MANIFEST to linker flags. Alright, let's edit pylzma's setup.py to add that linker flag for MSVC compiler:

    if isinstance(self.compiler, MSVCCompiler):
        # set flags only available when using MSVC
        ext.extra_link_args.append('/MANIFEST') # force linker to create manifest
        if COMPILE_DEBUG:
            ext.extra_compile_args.append('/Zi')
            ext.extra_compile_args.append('/MTd')
            ext.extra_link_args.append('/DEBUG')
        else:
            ext.extra_compile_args.append('/MT')

Voila, pylzma builds with MSVC fine now. I have tested building with python 2.4-2.7 32-bit and 2.6-2.7 64-bit:

08.02.2011  10:08            71 844 pylzma-0.4.3dev-py2.4-win32.egg
08.02.2011  10:09            71 480 pylzma-0.4.3dev-py2.5-win32.egg
08.02.2011  10:07            79 358 pylzma-0.4.3dev-py2.6-win-amd64.egg
08.02.2011  10:09            75 637 pylzma-0.4.3dev-py2.6-win32.egg
08.02.2011  10:08            79 259 pylzma-0.4.3dev-py2.7-win-amd64.egg
08.02.2011  10:09            75 540 pylzma-0.4.3dev-py2.7-win32.egg
               6 File(s)        453 118 bytes

As of general approach to any errors - it probably requires some knowledge and experience to understand what's behind them.

Upvotes: 2

Related Questions