CMAKE: Wrong linker (g++) is used in executable to CUDA library link

Question

I have a static CUDA-based library (mixture of .cu and .cpp files) which I am trying to link against a test file test.cu which will include a lib_file_1.cuh from the library. Thus I want to link my .cu executable to my CUDA library, and up until now g++ is used instead of nvcc in the linker step. How do I fix this? Because now I only get undefined reference to "__cudaRegisterLinkedBinary_name_..." type of errors due to the wrong linkage.

The following CMakeLists.txt files in root, root/my_library and root/test code should reproduce the issue.

root:

cmake_minimum_required(VERSION 3.17.3)
project(library_test LANGUAGES CUDA CXX)

if (MSVC)
    add_compile_options(/W4)
else()
    add_compile_options(-Wall -Wextra -fno-omit-frame-pointer -g)
endif()

add_subdirectory(my_library)
add_subdirectory(test)

set(CMAKE_VERBOSE_MAKEFILE ON)

target_link_libraries(tester PRIVATE my_library)

in root/my_library:

find_package(CUDA COMPONENTS THRUST REQUIRED)

add_library(my_library STATIC src/lib_file_1.cu)

target_include_directories(my_library PUBLIC 
    include
    ${CUDA_INCLUDE_DIRS}
)

target_compile_options(my_library
    PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler -Wno-pedantic -dc -gencode arch=compute_61,code=sm_61 --expt-relaxed-constexpr -Xlinker -dlink>
)

target_link_libraries(my_library
    PUBLIC ${CUDA_LIBRARIES}
)

in root/test:

add_executable(tester test.cu)

target_include_directories(tester PUBLIC
    ${CMAKE_SOURCE_DIR}/my_library
    )

Let for instance lib_file_1.cuh be:

#include <thrust/device_vector.h>

__host__ __device__ void foo(double &r);

lib_file_1.cu:

#include <thrust/device_vector.h>
#include "lib_file_1.cuh"

__host__ __device__ void foo(double &r)
{
     r = 0;
}

and test.cu:

#include "lib_file_1.cuh"

int main()
{
     double random_number;
     foo(random_number);
     return 0;
}

I have seen that the error is a potential issue as explained in 6.6.3 Implicit CUDA host code: https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#potential-separate-compilation-issues, which I have found several posts solving this for general commandline project building/manual makefile building. However, I have not found a good solution for cmake yet.

The following results when linking the CUDA executable tester using make VERBOSE=1 after running cmake -DCMAKE_BUILD_TYPE=debug .. inside root/debug:

/usr/bin/g++  CMakeFiles/tester.dir/test.cu.o -o tester  ../my_library/libmy_library.a /usr/local/cuda-10.2/lib64/libcudart_static.a -lpthread -ldl /usr/lib/x86_64-linux-gnu/librt.so -lcudadevrt -lcudart_static  -L"/usr/local/cuda-10.2/targets/x86_64-linux/lib/stubs" -L"/usr/local/cuda-10.2/targets/x86_64-linux/lib" -lrt -lpthread -ldl
CMakeFiles/tester.dir/test.cu.o: In function `__sti____cudaRegisterAll()':
/tmp/tmpxft_00005a8a_00000000-5_test.cudafe1.stub.c:20: undefined reference to `__cudaRegisterLinkedBinary_39_tmpxft_00005a8a_00000000_6_test_cpp1_ii_main'
../my_library/libmy_library.a(lib_file_1.cu.o): In function `__sti____cudaRegisterAll()':
/tmp/tmpxft_000057c7_00000000-5_lib_file_1.cudafe1.stub.c:20: undefined reference to `__cudaRegisterLinkedBinary_45_tmpxft_000057c7_00000000_6_lib_file_1_cpp1_ii__Z3fooRd'
collect2: error: ld returned 1 exit status
test/CMakeFiles/tester.dir/build.make:106: recipe for target 'test/tester' failed

Answer 1

Apparently, adding

set_target_properties(my_library PROPERTIES CUDA_SEPARABLE_COMPILATION ON)

to the CMakeLists.txt in root/my_library did the trick. I did try the option globally with

set(CUDA_SEPARABLE_COMPILATION ON)

in the CMakeLists.txt in the root directory, but that did not work. I dont know why.