CODEWITHSUNDEEP
pythoncgccclang
Rob
Rob

Reputation: 1131

Python C extension with openmp for OS X

I have created a python extension for a C program. In linux, using gcc, everything works correctly and the extension can be installed typing:

sudo python setup.py install

But when I try to use it in OS X:

GCC 4.7:

I have installed gcc 4.9 using macports and I have added this line in my setup.py file

import os
os.environ["CC"]="gcc-mp-4.9"

And when I type sudo python setup.py install

I obtain this error:

unrecognized command line option '-Wshorten-64-to-32'

I have been looking for the solution and everybody say "Use clang instead of gcc" to solve that issue.

Clang 3.8:

I have also installed clang 3.8 (3.5 is installed in os X but it doesn't have openmp) and I have modified the file setup.py:

import os
os.environ["CC"]="clang-mp-3.8"

And I obtain this error:

unknown argument: '-mno-fused-madd'

In some forums I have found a possible solution for this issue setting an empty value for CFLAGS:

sudo CFLAGS="" python setup.py install

But I obtain a new error:

library not found for -lgomp

I use -fopenmp but I do not why -fgomp is called. In some forums people say that I must use gcc instead of clang, so I am at the starting point again.

I would like to find a solution to easily isntall this extension in OS X because I would like to create an extension that can be easily installed by anyone.

Upvotes: 0

Views: 328

Answers (1)

Andrew Dalke
Andrew Dalke

Reputation: 15305

I had a similar problem. Python is built with clang and uses clang-specific CFLAGS:

>>> import sysconfig
>>> sysconfig.get_config_var("CFLAGS")
'-fno-strict-aliasing -fno-common -dynamic -arch x86_64
-arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing
-fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall
-Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g
-fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE'

Distutils copies this information to the "UnixCCCompiler" instance, used to make the extension. However, as you found out, -Wshorten-64-to-32 is clang specific.

My solution was to modify how distutils builds extensions. The following removes that option from the list of command-line arguments to pass to the compiler before calling the real compilation code. (You code may not need to be so complicated. I support several compilers and configurations.)

def _is_gcc(compiler):
    return "gcc" in compiler or "g++" in compiler

class build_ext_subclass( build_ext ):
    def build_extensions(self):
        c = self.compiler.compiler_type
        if c == "unix":
            compiler_args = self.compiler.compiler
            c = compiler_args[0]  # get the compiler name (argv0)
            if _is_gcc(c):
                names = [c, "gcc"]
                # Fix up a problem on older Mac machines where Python
                # was compiled with clang-specific options:
                #  error: unrecognized command line option '-Wshorten-64-to-32'
                compiler_so_args = self.compiler.compiler_so
                for args in (compiler_args, compiler_so_args):
                    if "-Wshorten-64-to-32" in args:
                        del args[args.index("-Wshorten-64-to-32")]

        build_ext.build_extensions(self)

Then tell setup() to use this new subclass to build extensions:

setup(name = ...
      cmdclass = {"build_ext": build_ext_subclass},
     )

Upvotes: 1

Related Questions