CODEWITHSUNDEEP
ccmakempiscalapack
gkaf
gkaf

Reputation: 94

Use of MPI::MPI_C alias

I am building ScaLAPACK (commit 7e4e07070a489686287c36ab473d21cf29a54bdd) using CMake (version 3.20.1). The find_package utility is used in CMakeLists.txt to define the libraries, inclusion directories, and flags required to compile with MPI. The ScaLAPACK library is linked with MPI using the alias MPI::MPI_C:

target_link_libraries( scalapack ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES} MPI::MPI_C)

I am building a shared library, so according to CMake default behavior, the resulting DSO should not have any entries in its RPATH. However, the installed library DSO maintains in its RUNPATH the entries:

$ readelf -d ~/opt/scalapack/lib/libscalapack.so | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64:/usr/lib64:/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib]

If the library is linked with using the libraries variable MPI_C_LIBRARIES defined by the find_package utility,

target_link_libraries( scalapack ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES} ${MPI_C_LIBRARIES})

then there is no RUNPATH entry in the installed DSO.

Looking further into the Makefiles generated by CMake, I noticed the following differences:

# File: <build directory>/CMakeFiles/scalapack.dir/DependInfo.cmake
...
set(CMAKE_Fortran_TARGET_INCLUDE_PATH
  "/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/include"
  "/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/include"
)
...

# File: <build directory>/CMakeFiles/scalapack.dir/Dflags.make
...
C_INCLUDES = -isystem /opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/include

C_FLAGS = -O3 -DNDEBUG -fPIC -fexceptions

Fortran_DEFINES = -DAdd_ -Dscalapack_EXPORTS

Fortran_INCLUDES = -I/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/include -I/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/include

Fortran_FLAGS = -fallow-argument-mismatch -O2 -DNDEBUG -O2 -fPIC -fexceptions
...


# File: <build directory>/CMakeFiles/scalapack.dir/link.txt

/opt/apps/resif/aion/2020b/epyc/software/GCCcore/10.2.0/bin/gfortran -fPIC -fallow-argument-mismatch -O2 -DNDEBUG -O2 -Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib -Wl,--enable-new-dtags -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib -L/usr/lib64 -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib -shared -Wl,-soname,libscalapack.so.2.2 -o lib/libscalapack.so.2.2.1 @CMakeFiles/scalapack.dir/objects1.rsp  -Wl,-rpath,/home/users/gkafanas/opt/openblas/lib64: /home/users/gkafanas/opt/openblas/lib64/libopenblas.so -lpthread -lm -ldl /home/users/gkafanas/opt/openblas/lib64/libopenblas.so -lpthread -lm -ldl /opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib/libmpi.so


# File: <build directory>/CMakeFiles/scalapack.dir/DependInfo.cmake
...
set(CMAKE_Fortran_TARGET_INCLUDE_PATH
)
...

# File: <build directory>/CMakeFiles/scalapack.dir/Dflags.make
...
C_INCLUDES = 

C_FLAGS = -O3 -DNDEBUG -fPIC

Fortran_DEFINES = -DAdd_ -Dscalapack_EXPORTS

Fortran_INCLUDES = 

Fortran_FLAGS = -fallow-argument-mismatch -O2 -DNDEBUG -O2 -fPIC
...


# File: <build directory>/CMakeFiles/scalapack.dir/link.txt

/opt/apps/resif/aion/2020b/epyc/software/GCCcore/10.2.0/bin/gfortran -fPIC -fallow-argument-mismatch -O2 -DNDEBUG -O2 -shared -Wl,-soname,libscalapack.so.2.2 -o lib/libscalapack.so.2.2.1 @CMakeFiles/scalapack.dir/objects1.rsp  -Wl,-rpath,/home/users/gkafanas/opt/openblas/lib64: /home/users/gkafanas/opt/openblas/lib64/libopenblas.so -lpthread -lm -ldl /home/users/gkafanas/opt/openblas/lib64/libopenblas.so /opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib/libmpi.so -lpthread -lm -ldl /opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib/libmpi.so

So it seems that the alias sets some system options that are not necessary.

I have 2 questions:

  1. What is the alias MPI::MPI_C and how it should be used.
  2. Why the alias sets so may parameters when someone might expect that it will set only the variables used to link the ScaLAPACK DSO with MPI.

Upvotes: 0

Views: 234

Answers (2)

gkaf
gkaf

Reputation: 94

I had to revisit this issue recently. It seems that the find_package utility populates some variables with useless values:

$ cmake -LAH -S . -B build
...
// MPI C linker flags
MPI_C_LINK_FLAGS:STRING=-Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib -Wl,--enable-new-dtags -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib -L/usr/lib64 -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib
...
// MPI Fortran linker flags
MPI_Fortran_LINK_FLAGS:STRING=-Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64 -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib -Wl,--enable-new-dtags -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib -L/usr/lib64 -L/mnt/irisgpfs/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib
...

These are variables contain linking options that add the spurious paths in the RUNPATH of the resulting library. The link options are added to the target libraries by linking with alias targets.

The targets MPI::MPI_C and MPI::MPI_Fortran are alias libraries, a type of alias target. Alias targets refer to an actual target, in this case the external libraries for MPI, and can also define additional properties, such as link options. It seems that in this case the link flags are added in the target link options for MPI::MPI_C and MPI::MPI_Fortran, so linking with these alias libraries we inadvertently introduce the spurious link options.

The solution suggested by @VictorEijkhout uses directly the variables defined by the find_package to setup the target properties, and thus avoids unnecessary side effects.

Upvotes: 0

Victor Eijkhout
Victor Eijkhout

Reputation: 5810

I'll answer your question indirectly. Since CMake is so little standardized, I have no idea what MPI::MPI_C is or whether it can be made to work. Here is what I use:

find_package( MPI )

add_executable( ${PROJECT_NAME} ${PROJECT_NAME}.c )
target_include_directories(
        ${PROJECT_NAME} PUBLIC
        ${MPI_C_INCLUDE_DIRS} ${CMAKE_CURRENT_SOURCE_DIR} )
target_link_libraries(
        ${PROJECT_NAME} PUBLIC
        ${MPI_C_LIBRARIES} )

This is for a single source file, but it can be adapted to your own uses.

Upvotes: 1

Related Questions