Telling GCC to *not* link libgomp so it links libiomp5 instead

Question

I need to figure out a compiler/linker directive I can feed into gcc so that it won't automatically link libgomp when -fopenmp is specified.

The reason is that I'm trying to build against Intel's MKL BLAS. MKL requires adding a separate Intel library to handle multithreading (e.g., libmkl_intel_thread or libmkl_gnu_thread). The library for linking MKL against libgomp, however, is not available on every operating system, including mine. This forces me to link libmkl_intel_thread, which in turn must link against libiomp5.

While I am able to build my package, some binaries are linked against both libgomp and libiomp5. I'm not positive this is causing problems, but there have been a few crashes, the linkage-combination is suspicious, and even if it isn't causing crashes its certainly a terrible inefficiency.

I'm trying to do this with gcc 4.9.1.

Avoiding -fopenmp is, unfortunately, not an option. The reason is that this is for compiling a rather large package comprised of several sub-packages, whose Makefiles are not in the greatest shape, and where additional packages from other sources (plug-ins) may be compiled later. Forcing a universal compiler/linker directive isn't difficult. Turning on --enable-openmp, however, activates both -fopenmp, and defines that are used to trigger code related to multi-threading. Trying to separate the three (--enable-openmp, -fopenmp, and code linked to --enable-openmp) isn't feasible.

I've looked through manual pages, and I don't see any directive for gcc that would allow selection of an openmp library. Intel's forums have a very old discussion in which they suggest specifying a static library immediately after -fopenmp followed by --as-needed. Which seems pretty rickety, and also has a lot of potential to interfere with plugin packages. llvm-openmp seems to have considered a -fopenmp=libiomp5 directive at one point, but it seems to have been dropped in the 3.5 release and I'm trying to use gcc anyway.

Thanks.

Answer 1

I think I have an answer at this point; I've had several exchanges with Intel folks, and I want to share the result. This is a mixture of some of their suggestions and what I came up with on my own:

1. The short answer is, you can't. Gcc wants to force the use of libgomp at the linker stage. If libiomp is also linked, then both libraries will be linked. Which one will be called? I don't know.

2. The longer answer is that on some distributions, it may be possible to change gcc's default behavior (adding libgomp whenever -fopenmp is set) by creating a custom libgomp.spec or altering the libgomp.spec installed with gcc. On my distribution (homebrew), this was not feasible; the "libgomp.spec" file is empty, and spec's for libgomp are built-in to gcc. All of those would have to be overridden. And this would have to be redone whenever gcc is updated.

3. It may be possible on some operating systems to replace every copy and link to libgomp, to a symlink to libiomp5. The binary will then have multiple links leading to the same library, under two different names. What will happen then? I don't know.

4. What I ended-up doing, is moving from gcc to the clang-omp implementation of llvm. It uses libiomp5 unless told otherwise. My concern about this was that part of my project uses fortran, and there's no llvm fortran compiler. It turns out, though, that even when -fopenmp is given to gfortran, so long as llvm is ultimately doing the linking it will wipe-out any references to libgomp and replace them with libiomp5. clang-omp also may have an option to select the omp library with -fopenmp=[libiomp5|libgomp], but I was not able to get this working consistently. Anyway, the clang-omp implementation of llvm 3.5 covers almost all of the openmp spec, and so far it doesn't appear that anything was lost in the switch. In fact performance improved.

5. I did, for the record, experiment with using gfortran as an llvm frontend using dragonegg. This book was not worth the candle. Dragonegg isn't compatible with gcc 4.9 so it forces gcc 4.8. It was difficult to setup; appears that it would be difficult to maintain as versions change; the llvm folks are not sure how much support dragonegg will have moving forward; and in all events performance was inferior to just using llvm.

6. The question that drove me here, was how to get a package with C and fortran components, that uses OpenMP, compiled against MKL, which the MKL libraries for my OS are hardlinked against iomp5 and won't accept gomp. The answer to that is that the only viable option was to move from gcc to clang-omp.

7. This does leave the question, "is iomp5 'drop-in compatible' with gcc 4.9", as claimed on the OpenMP website. The answer is simply, "no," iomp5 and gcc 4.9 will not work with each other --- at least without substantial modifications to one's toolchain, for which no guidance or documentation is available, and it is not clear whether anyone has done this successfully.

Answer 2

I'm a technical support engineer on the Intel MKL team. This post has recently come to our attention. The threading layer of Intel MKL does indeed require libiomp5, ie. the Intel OpenMP runtime library. To properly link with libiomp5 using GCC, the code needs to be compiled without using '-fopenmp'. And then the link line needs to explicitly include libmkl_intel_thread and libiomp5.

The current MKL documentation does not provide enough explanations on this usage model. And the MKL link line advisor is outright broken. I'm sorry for all the inconvenience and confusions. We will get the link line advisor fixed as soon as possible, and will improve the User's Guide to better help GCC users on OS X.

Answer 3

GCC does not support linking against the Intel OpenMP runtime library. GCC's internal code transformer translates OpenMP directives into ligomp-specific calls and those have a way different API than the one exposed by libiomp. Also, mixing two separate OpenMP runtimes into one executable (or into a single process, if OpenMP-enabled modules are being loaded dynamically) is a recipe for disaster. That's the reason why MKL's multithreaded driver comes in two flavours - an Intel one and a GNU one. That the latter is missing on some machines is probably a defect of the installation.

Edit: Apparently Intel OpenMP runtime provides a GNU compatibility layer, which means that it could possibly be used as a drop-in replacement of libgomp. At least the symbols are there:

$ nm libiomp5.a | sort | grep GOMP_
0000000000000000 T GOMP_barrier@@VERSION
0000000000000000 T GOMP_barrier@GOMP_1.0
0000000000000000 T __kmp_api_GOMP_barrier
0000000000000000 T __kmp_api_GOMP_barrier_10_alias
...

In that case, what you have to do is:

• keep -fopenmp while compiling the code so that GCC would recognise the OpenMP pragmas and transform the code into the corresponding calls into libgomp;
• If GCC is used to link the executable or shared library, do not pass it the -fopenmp option during the linking phase; instead, pass -L/path/to/libiomp5 -liomp5;
• If GNU ld is used to link the executable/module, replace -lgomp with -liomp5.

If not possible to do the aforementioned changes, the thread on the Intel's forums makes some sense because of the way the linker resolves link-time symbol references, though it is really more of a hack. Passing -Wl,--as-needed forces GNU ld to not emit DT_NEEDED tags for any library that follows it on the command line unless that library satisfies an undefined symbol reference, the assumption being that the GCC driver will insert -lgomp somewhere after the user-supplied options. The idea is to prevent libgomp from being linked with the executable even when there are no unresolved references to GOMP_..., which normally should not be the case since all references, even those from dynamically loaded modules, should be satisfied by libiomp5. Preventing libgomp from being loaded by the RTLD is essential since there are some constructor routines in it that are called no matter if symbols are being imported or not and those do things that might interfere with IOMP.

The linker trick won't work on non-ELF systems like OS X. The Mach-O link editor does not support --as-needed though there might be a different mechanism to achieve the same result on that OS.