Hossein
Reputation: 25934
CMake cant find the external libraries when subdirectory is used to build dependencies for each target
I have a set of libraries and their respective tests, and they are organized in the following fashion:
-Lib1
-Lib1_Test
-Lib2
-Lib2_Test
-Lib3
-Lib3_Test
....
and so on. some of these libs depend on others, for example Lib1 depends on Lib2, and Lib3 depends on Lib1. I can easily build each target separately and all is fine. However, When I try to build them in such a way that each target first builds its dependencies and then itself I face an issue here.
For this very purpose, I used 'add_subdirectory` for each dependenant target and I also check if the target is already built or not so only if its not, we include it and build it.
This is how one of my CMakeLists.txt look like :
cmake_minimum_required(VERSION 3.11)
project(FV)
set(CMAKE_CXX_STANDARD 17)
set( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC" )
find_package(Torch REQUIRED)
find_package(OpenCV REQUIRED)
find_package(Protobuf REQUIRED)
if ("${TORCH_LIB_DIRS}" STREQUAL "")
set(TORCH_LIB_DIRS /home/rika/libtorch-cxx11-abi-shared-with-deps-1.6.0+cpu/libtorch/lib)
endif()
get_filename_component(PARENT_DIR ${PROJECT_SOURCE_DIR} DIRECTORY)
#if (DLL)
add_definitions(-D_FV_BUILD_DLL)
set(CMAKE_INSTALL_PREFIX ${PARENT_DIR}/built_stuff)
if(NOT TARGET AntiSpoofer)
add_subdirectory(${PARENT_DIR}/AntiSpoofer ${PARENT_DIR}/AntiSpoofer/build)
endif()
if(NOT TARGET Detector)
add_subdirectory(${PARENT_DIR}/Detector ${PARENT_DIR}/Detector/build)
endif()
if(NOT TARGET Hashing)
add_subdirectory(${PARENT_DIR}/Hashing ${PARENT_DIR}/Hashing/build)
endif()
include_directories(${PARENT_DIR}/Dependencies/include ${Protobuf_INCLUDE_DIRS} ${TORCH_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS})
set(FV_H ${PARENT_DIR}/Dependencies/include/FV/FV.h)
set(Config_H ${PARENT_DIR}/Dependencies/include/messages/Config.pb.h)
set(TorchSerializer_H ${PARENT_DIR}/Dependencies/include/messages/TorchSerializer.pb.h)
set(TorchSerializer_PY ${PARENT_DIR}/Dependencies/include/messages/TorchSerializer_pb2.py)
set(FV_SRC ./FV.cpp)
set(Config_SRC ${PARENT_DIR}/Dependencies/include/messages/Config.pb.cc)
set(TorchSerializer_SRC ${PARENT_DIR}/Dependencies/include/messages/TorchSerializer.pb.cc)
LINK_DIRECTORIES(${TORCH_LIB_DIRS})
LINK_DIRECTORIES(${PARENT_DIR}/AntiSpoofer/build)
LINK_DIRECTORIES(${PARENT_DIR}/Blinker/build)
LINK_DIRECTORIES(${PARENT_DIR}/Detector/build)
LINK_DIRECTORIES(${PARENT_DIR}/Hashing/build)
add_library(
FV
SHARED
${FV_SRC}
${Config_SRC}
${TorchSerializer_SRC}
)
target_link_directories(FV PUBLIC ${PARENT_DIR}/built_stuff/lib)
# Link
target_link_libraries(FV ${OpenCV_LIBS})
target_link_libraries(FV ${TORCH_LIB_DIRS}/libc10.so)
target_link_libraries(FV ${TORCH_LIB_DIRS}/libtorch_cpu.so)
target_link_libraries(FV Hashing)
target_link_libraries(FV Blinker)
target_link_libraries(FV AntiSpoofer)
target_link_libraries(FV Detector)
# add FV include
install(TARGETS FV LIBRARY DESTINATION lib)
install(FILES ${FV_H} DESTINATION include/FV)
# add protobuf related headers/srcs for c++ and python
install(FILES ${Config_H} DESTINATION include/messages)
install(FILES ${TorchSerializer_H} DESTINATION include/messages)
install(FILES ${Config_SRC} DESTINATION src/messages)
install(FILES ${TorchSerializer_SRC} DESTINATION src/messages)
install(FILES ${TorchSerializer_PY} DESTINATION Python)
when I try to call this very cmakelists like this :
cmake -DCMAKE_PREFIX_PATH="/home/rika/libtorch-cxx11-abi-shared-with-deps-1.6.0+cpu/libtorch/share/cmake" ..
everything builds just fine and all targets seem to be able to get and use the passed CMAKE_PREFIX_PATH that I send here. However, just after this, if I cd into the FV_Test which by the way is given below, it fails with the error messages indicating the failure of CMake in finding torch! which is clearly specified in the argument when calling CMake. This is the error message I get after trying to build FV_Test:
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /usr/local (found version "3.4.10")
-- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found version "3.13.0")
-- Found Threads: TRUE
-- Found Torch: /home/rika/libtorch-cxx11-abi-shared-with-deps-1.6.0+cpu/libtorch/lib/libtorch.so
-- Using CMake version: 3.18.2
-- Compiling dlib version: 19.21.0
-- Found X11: /usr/include
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found system copy of libpng: /usr/lib/x86_64-linux-gnu/libpng.so;/usr/lib/x86_64-linux-gnu/libz.so
-- Found system copy of libjpeg: /usr/lib/x86_64-linux-gnu/libjpeg.so
-- Searching for BLAS and LAPACK
-- Searching for BLAS and LAPACK
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'cblas'
-- No package 'cblas' found
-- Checking for module 'lapack'
-- Found lapack, version 3.10.3
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found Intel MKL BLAS/LAPACK library
-- Looking for sgesv
-- Looking for sgesv - found
-- Looking for sgesv_
-- Looking for sgesv_ - found
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) (Required is at least version "7.5")
-- DID NOT FIND CUDA
-- Disabling CUDA support for dlib. DLIB WILL NOT USE CUDA
-- C++11 activated.
-- Configuring done
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_COMPILE_DEFINITIONS>
Target "torch_cpu" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_COMPILE_DEFINITIONS>
Target "torch" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch_cpu" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch" not found.
CMake Error:
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_SYSTEM_INCLUDE_DIRECTORIES>
Target "torch_cpu" not found.
CMake Error:
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch_cpu" not found.
CMake Error:
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_SYSTEM_INCLUDE_DIRECTORIES>
Target "torch" not found.
CMake Error:
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_COMPILE_OPTIONS>
Target "torch_cpu" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_COMPILE_OPTIONS>
Target "torch" not found.
CMake Warning at CMakeLists.txt:49 (add_executable):
Cannot generate a safe runtime search path for target FV_test because files
in some directories may conflict with libraries in implicit directories:
runtime library [libpng16.so.16] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
Some of these libraries may not be found correctly.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_COMPILE_DEFINITIONS>
Target "torch_cpu" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_COMPILE_DEFINITIONS>
Target "torch" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch_cpu,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch_cpu" not found.
CMake Error at CMakeLists.txt:58 (target_link_libraries):
Error evaluating generator expression:
$<TARGET_PROPERTY:torch,INTERFACE_INCLUDE_DIRECTORIES>
Target "torch" not found.
CMake Warning at /home/rika/Desktop/LibtorchPort/FV/CMakeLists.txt:68 (add_library):
Cannot generate a safe runtime search path for target FV because files in
some directories may conflict with libraries in implicit directories:
runtime library [libpng16.so.16] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
Some of these libraries may not be found correctly.
CMake Warning at /home/rika/Desktop/LibtorchPort/AntiSpoofer/CMakeLists.txt:57 (add_library):
Cannot generate a safe runtime search path for target AntiSpoofer because
files in some directories may conflict with libraries in implicit
directories:
runtime library [libpng16.so.16] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
Some of these libraries may not be found correctly.
CMake Warning at /home/rika/Desktop/LibtorchPort/Blinker/CMakeLists.txt:46 (add_library):
Cannot generate a safe runtime search path for target Blinker because files
in some directories may conflict with libraries in implicit directories:
runtime library [libpng16.so.16] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/rika/anaconda3/lib
Some of these libraries may not be found correctly.
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.
And this is the CMakeLists.txt that I'm using for FV_Test.
cmake_minimum_required(VERSION 3.11)
project(FV_Test)
set(CMAKE_CXX_STANDARD 17)
find_package(OpenCV REQUIRED)
find_package(Protobuf REQUIRED)
if ("${TORCH_LIB_DIRS}" STREQUAL "")
set(TORCH_LIB_DIRS /home/rika/libtorch-cxx11-abi-shared-with-deps-1.6.0+cpu/libtorch/lib)
endif()
get_filename_component(PARENT_DIR ${PROJECT_SOURCE_DIR} DIRECTORY)
if(NOT TARGET FV)
add_subdirectory(${PARENT_DIR}/FV ${PARENT_DIR}/FV/build)
endif()
set(CMAKE_INSTALL_PREFIX ${PARENT_DIR}/built_stuff)
include_directories(${PARENT_DIR}/Dependencies/include ${OpenCV_INCLUDE_DIRS} ${Protobuf_INCLUDE_DIRS}) #
LINK_DIRECTORIES(${TORCH_LIB_DIRS})
LINK_DIRECTORIES(${PARENT_DIR}/AntiSpoofer/build)
LINK_DIRECTORIES(${PARENT_DIR}/Blinker/build)
LINK_DIRECTORIES(${PARENT_DIR}/FV/build)
add_executable(FV_test ./FV_Test.cpp )
target_link_directories(FV_test PUBLIC ${PARENT_DIR}/AntiSpoofer/build)
target_link_directories(FV_test PUBLIC ${PARENT_DIR}/Blinker/build)
target_link_directories(FV_test PUBLIC ${PARENT_DIR}/FV/build)
# Link
target_link_libraries(FV_test ${OpenCV_LIBS})
target_link_libraries(FV_test ${Protobuf_LIBRARIES})
target_link_libraries(FV_test FV)
target_link_libraries(FV_test Blinker)
install(TARGETS FV_test DESTINATION bin)
Why am I seeing this behavior and how can I solve this?
Upvotes: 3
Views: 2319
Answers (1)
Quilan
Reputation: 21
If the problem is really depicted in the following comment:
As far as I understand, the core of the problem is: 1. In subdirectory use find_package(Torch REQUIRED) and then link some library with a Torch library via target_link_libraries(mylib ${TORCH_LIBRARIES}). 2. Linking with that library in the outer directory: target_link_libraries(myexe mylib). For some (unknown) reason, during the last link CMake tries to evaluate generator expressions for torch_cpu. But this is an IMPORTED target, which is created in the subdirectory, so it cannot be accessed from the outer directory. – Tsyvarev Sep 27 '20 at 8:15
I ran into the same problem with Pytorch. The fix that worked for me was to link the torch libraries privately: target_link_libraries(mylib PRIVATE ${TORCH_LIBRARIES}").
Now don't ask my why that works I wouldn't be able to answer that.
Upvotes: 2
Related Questions
- CMake Linking library with full path and with target_include_directories
- Pointing cmake to a directory that contains a dependency
- CMake can't find include files of libraries
- CMAKE : Link a library from a parent directory
- Cmake: target_link_libraries propagation of include directories
- How to use add_dependency with CMake to non-subdirectories?
- CMake: cannot find source file on add_library of sibling library
- how do i build libraries in subdirectories using cmake?
- CMake: Linking to an imported library with dependencies fails
- CMake subdirectories dependency