CODEWITHSUNDEEP
pythoncbuildnumpyintel-mkl
MattDMo
MattDMo

Reputation: 102862

Error building NumPy with MKL on OSX with Python 3.3

Background

Using a 2012 iMac with a 2nd-gen Core i7 processor, I'm trying to build NumPy 1.7.0 (and eventually SciPY) on OSX 10.8.3 linked to the MKL included in the evaluation versions of Intel's C++ Composer XE 2013 and Fortran Composer XE 2013 for OSX. I am following this article on Intel's site (loosely adapted, as it's not very specific in some cases) for settings, compiler flags, etc.

Problem

Using either:

$ python3 setup.py config build --compiler=intelem --fcompiler=intelem

or

$ python3 setup.py config --compiler=intelem --fcompiler=intelem build_clib --compiler=intelem --fcompiler=intelem build_ext --compiler=intelem --fcompiler=intelem

I get the same error:

running build_clib
customize IntelEM64TCCompiler
customize IntelEM64TCCompiler using build_clib
running build_ext
customize IntelEM64TCCompiler
customize IntelEM64TCCompiler using build_ext
building 'numpy.core._dummy' extension
compiling C sources
C compiler: icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include

compile options: '-Inumpy/core/include -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/src/multiarray -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/src/umath -c'
icc: numpy/core/src/dummymodule.c
icc: command line warning #10121: overriding '-marchcorei7' with '-xhost'
icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include -shared build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so
icc: command line warning #10121: overriding '-marchcorei7' with '-xhost'
icc: command line warning #10006: ignoring unknown option '-shared'
Undefined symbols for architecture x86_64:
  "_PyModule_Create2", referenced from:
      _PyInit__dummy in dummymodule.o
  "_main", referenced from:
      start in crt1.10.6.o
ld: symbol(s) not found for architecture x86_64
icc: command line warning #10121: overriding '-marchcorei7' with '-xhost'
icc: command line warning #10006: ignoring unknown option '-shared'
Undefined symbols for architecture x86_64:
  "_PyModule_Create2", referenced from:
      _PyInit__dummy in dummymodule.o
  "_main", referenced from:
      start in crt1.10.6.o
ld: symbol(s) not found for architecture x86_64
error: Command "icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include -shared build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so" failed with exit status 1

My LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are both:

/opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib

my DYLD_FRAMEWORK_PATH is:

/opt/local/Library/Frameworks:/Library/Frameworks:/System/Library/Frameworks

and my site.cfg is:

[DEFAULT]
library_dirs = /opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib
include_dirs = /opt/intel/include:/opt/intel/include/intel64:/opt/intel/mkl/include:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m:/opt/local/include:/usr/local/include:/usr/include

[mkl]
library_dirs = /opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib
include_dirs = /opt/intel/include:/opt/intel/include/intel64:/opt/intel/mkl/include:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m:/opt/local/include:/usr/local/include:/usr/include
mkl_libs = mkl_rt
lapack_libs = mkl_lapack

Symlinks to icc and ifort are in /usr/bin and are functional. The contents of dummymodule.c are available in the numpy repository on GitHub. I'm not a C programmer, so I can't immediately see what's wrong. I've gotten this error using both MacPorts python 3.3.0 (using the above settings) and with the python.org 3.3.0, with settings modified to point to it instead of /opt/local.

Plea

Can anyone help figure this out?

EDIT

I swapped out -shared for -dynamiclib, and am now getting:

building extension "numpy.random.mtrand" sources
C compiler: icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64

compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c'
icc: _configtest.c
icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 _configtest.o -o _configtest
_configtest
failure.
removing: _configtest.c _configtest.o _configtest
building data_files sources
build_src: building npy-pkg config files
running build_py
copying numpy/version.py -> build/lib.macosx-10.8-x86_64-3.3/numpy
copying build/src.macosx-10.8-x86_64-3.3/numpy/__config__.py -> build/lib.macosx-10.8-x86_64-3.3/numpy
copying numpy/distutils/intelccompiler.py -> build/lib.macosx-10.8-x86_64-3.3/numpy/distutils
copying build/src.macosx-10.8-x86_64-3.3/numpy/distutils/__config__.py -> build/lib.macosx-10.8-x86_64-3.3/numpy/distutils
copying numpy/distutils/fcompiler/intel.py -> build/lib.macosx-10.8-x86_64-3.3/numpy/distutils/fcompiler
running build_clib
customize IntelEM64TCCompiler
customize IntelEM64TCCompiler using build_clib
running build_ext
customize IntelEM64TCCompiler
customize IntelEM64TCCompiler using build_ext
building 'numpy.core._dummy' extension
compiling C sources
C compiler: icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64

compile options: '-Inumpy/core/include -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c'
icc: numpy/core/src/dummymodule.c
icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -dynamiclib build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so
Undefined symbols for architecture x86_64:
  "_PyModule_Create2", referenced from:
      _PyInit__dummy in dummymodule.o
ld: symbol(s) not found for architecture x86_64
Undefined symbols for architecture x86_64:
  "_PyModule_Create2", referenced from:
      _PyInit__dummy in dummymodule.o
ld: symbol(s) not found for architecture x86_64
error: Command "icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -dynamiclib build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so" failed with exit status 1

So, I get a line that says failure. after _configtest (not sure what that's about) and the same error(s) down below. Thoughts?

Upvotes: 0

Views: 1532

Answers (2)

icylord
icylord

Reputation: 56

I recommend you change file numpy-1.7.0/numpy/distutils/intelccompiler.py line 37 to

icc -m64 -fPIC -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -undefined dynamic_lookup -bundle -DMKL_ILP64

This work for me where i have this compile error before

Upvotes: 1

autistic
autistic

Reputation: 15642

icc: command line warning #10006: ignoring unknown option '-shared'

Here's a problem. Your compiler is trying to build an executable, even though it's been told to build a shared library. According to the icc documentation, -shared is only available on Linux systems. Since yours is a MacOSX system, I think you'd want to use -dynamiclib instead.

edit: I suspect you're missing a link to pylib, in: icc -m64 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -dynamiclib build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so.

The missing link causes this:

Undefined symbols for architecture x86_64:
  "_PyModule_Create2", referenced from:
      _PyInit__dummy in dummymodule.o
ld: symbol(s) not found for architecture x86_64

You need to add -lpythonX.Y, where X.Y represents the version of Python you're linking against. If this file exists, but the linker can't find it, you'll need to tell the linker where it is by using a -Llink/search/directory (eg. -Llib/) directive. If the file doesn't exist, you need to build it.

Upvotes: 3

Related Questions