Relocation R_X86_64_32S against `.bss' can not be used when making a PIE linking error

Question

I'm trying to compile AccNEAT project with CUDA support. It's working fine when I compile it without CUDA support. However, when I compile with CUDA support, I get linker errors. To compile the project, my environment is Ubuntu 18.04 LTS 64 bit with GCC-4.8 and NVCC 6.0.

Linker error:

/usr/bin/x86_64-linux-gnu-ld: obj/cu/network/cuda/cudanetwork.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/maze/mazeevaluator.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/static/staticevaluator.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:49: recipe for target 'neat' failed
make: *** [neat] Error 1

The compiler suggests adding a -fPIC option, but I don't know exactly where to put it in the Makefile. I tried to add it using a -XCompiler option like -XCompiler "-fPIC...", but then when I compile I get this error:

obj/cu/cxx/experiments/maze/mazeevaluator.o: In function `NEAT::create_config(NEAT::Config*&, unsigned long&)':
/home/ner/Projekty/accneat-master-cuda/src/experiments/maze/mazeevaluator.cxx:158: undefined reference to `NEAT::find_resource(std::string const&)'
/home/ner/Projekty/accneat-master-cuda/src/experiments/maze/mazeevaluator.cxx:158: undefined reference to `NEAT::parse_map(std::string)'
collect2: error: ld returned 1 exit status
Makefile:49: recipe for target 'neat' failed
make: *** [neat] Error 1

What does the 'relocation' error mean and is there way to fix this problem?

My Makefile:

include Makefile.conf

CC_CUDA=nvcc -DENABLE_CUDA ${NVCC_FLAGS} -arch=sm_13 --debug --compiler-bindir ${PFM_NVCC_CCBIN} -Xcompiler "${OPT} ${INCLUDES} ${OPENMP} -I/usr/local/cuda/include "

INCLUDES=$(patsubst %,-I%,$(shell find src -type d))
SOURCES=$(shell find src -name "*.cpp")
CXX_SOURCES=$(shell find src -name "*.cxx")

OBJECTS=${SOURCES:src/%.cpp=obj/cpp/%.o}

LIBS=-lgomp
DEFINES=

ifeq (${ENABLE_CUDA}, true)
    CUDA_SOURCES=$(shell find src -name "*.cu")
    CUDA_OBJECTS=${CUDA_SOURCES:src/%.cu=obj/cu/%.o}
    CUDA_OBJECTS+=${CXX_SOURCES:src/%.cxx=obj/cu/cxx/%.o}
    LIBS+=-lcudart
    DEFINES+=-DENABLE_CUDA
else
    OBJECTS+=${CXX_SOURCES:src/%.cxx=obj/cpp/cxx/%.o}
endif

DEPENDS=${OBJECTS:%.o=%.d}
DEPENDS+=${CUDA_OBJECTS:%.o=%.d}

ifeq (${DEVMODE}, true)
    OPT=-O0
    #OPENMP=-fopenmp
    MISC_FLAGS=
    NVCC_FLAGS=-G -g
else
    OPT=-O3
    OPENMP=-fopenmp
    MISC_FLAGS=-Werror
endif

CC_FLAGS=-Wall ${DEFINES} ${MISC_FLAGS} ${PROFILE} ${INCLUDES} ${OPENMP} ${OPT} -c -g -gdwarf-3
.PHONY: clean default

default: ./neat

clean:
    rm -rf obj
    rm -f ./neat
    rm -f src/util/std.h.gch

./neat: ${OBJECTS} ${CUDA_OBJECTS}
    g++ ${PROFILE} ${OBJECTS} ${CUDA_OBJECTS} ${PFM_LD_FLAGS} ${LIBS} -o $@

src/util/std.h.gch: src/util/std.h Makefile.conf Makefile
    g++ ${CC_FLAGS} -std=c++11 $< -o $@

ifeq (${ENABLE_CUDA}, true)

obj/cu/cxx/%.o: src/%.cxx Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    ${CC_CUDA} -c -x cu $< -o $@

obj/cu/cxx/%.d: src/%.cxx
    @mkdir -p $(dir $@)
    @${CC_CUDA} -M -x cu $< > $@.tmp
    @cat $@.tmp | sed 's,.*\.o[[:space:]]*:,$@:,g' | sed 's,\.d,\.o,' > $@
    @rm $@.tmp

obj/cu/%.o: src/%.cu Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    ${CC_CUDA} -c $< -o $@

obj/cu/%.d: src/%.cu
    @mkdir -p $(dir $@)
    @${CC_CUDA} -M $< > $@.tmp
    @cat $@.tmp | sed 's,.*\.o[[:space:]]*:,$@:,g' | sed 's,\.d,\.o,' > $@
    @rm $@.tmp

else
obj/cpp/cxx/%.o: src/%.cxx Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    g++ ${CC_FLAGS} -std=c++98 -MMD $< -o $@
endif

obj/cpp/%.o: src/%.cpp Makefile.conf Makefile src/util/std.h.gch
    @mkdir -p $(shell dirname $@)
    g++ ${CC_FLAGS} -std=c++11 -MMD $< -o $@

-include ${DEPENDS}

(I have added small changes to make it compile like include nvcc dir path).

Build command:

g++  obj/cpp/experiments/maze/maze.o obj/cpp/experiments/static/regex.o obj/cpp/experiments/static/cfg.o obj/cpp/experiments/static/xor.o obj/cpp/experiments/static/sequence.o obj/cpp/experiments/experiment.o obj/cpp/innovgenome/innovation.o obj/cpp/innovgenome/trait.o obj/cpp/innovgenome/innovnodegene.o obj/cpp/innovgenome/innovlinkgene.o obj/cpp/innovgenome/innovgenome.o obj/cpp/innovgenome/innovgenomemanager.o obj/cpp/genomemanager.o obj/cpp/population.o obj/cpp/network/cpu/cpunetwork.o obj/cpp/species/speciesorganism.o obj/cpp/species/species.o obj/cpp/species/speciespopulation.o obj/cpp/util/timer.o obj/cpp/util/rng.o obj/cpp/util/util.o obj/cpp/util/resource.o obj/cpp/util/map.o obj/cpp/main.o obj/cpp/organism.o obj/cpp/neat.o obj/cu/network/cuda/cudanetwork.o obj/cu/cxx/experiments/maze/mazeevaluator.o obj/cu/cxx/experiments/static/staticevaluator.o  -lgomp -lcudart -o neat

EDIT 1: added full build log:

nerexis@nerexis-GE70-0NC-GE70-0ND:~/Projekty/accneat-master-cuda$ make clean
rm -rf obj
rm -f ./neat
rm -f src/util/std.h.gch
nerexis@nerexis-GE70-0NC-GE70-0ND:~/Projekty/accneat-master-cuda$ make
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 src/util/std.h -o src/util/std.h.gch
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/maze/maze.cpp -o obj/cpp/experiments/maze/maze.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/static/regex.cpp -o obj/cpp/experiments/static/regex.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/static/cfg.cpp -o obj/cpp/experiments/static/cfg.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/static/xor.cpp -o obj/cpp/experiments/static/xor.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/static/sequence.cpp -o obj/cpp/experiments/static/sequence.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/experiments/experiment.cpp -o obj/cpp/experiments/experiment.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/innovation.cpp -o obj/cpp/innovgenome/innovation.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/trait.cpp -o obj/cpp/innovgenome/trait.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/innovnodegene.cpp -o obj/cpp/innovgenome/innovnodegene.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/innovlinkgene.cpp -o obj/cpp/innovgenome/innovlinkgene.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/innovgenome.cpp -o obj/cpp/innovgenome/innovgenome.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/innovgenome/innovgenomemanager.cpp -o obj/cpp/innovgenome/innovgenomemanager.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/genomemanager.cpp -o obj/cpp/genomemanager.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/population.cpp -o obj/cpp/population.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/network/cpu/cpunetwork.cpp -o obj/cpp/network/cpu/cpunetwork.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/species/speciesorganism.cpp -o obj/cpp/species/speciesorganism.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/species/species.cpp -o obj/cpp/species/species.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/species/speciespopulation.cpp -o obj/cpp/species/speciespopulation.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/util/timer.cpp -o obj/cpp/util/timer.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/util/rng.cpp -o obj/cpp/util/rng.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/util/util.cpp -o obj/cpp/util/util.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/util/resource.cpp -o obj/cpp/util/resource.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/util/map.cpp -o obj/cpp/util/map.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/main.cpp -o obj/cpp/main.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/organism.cpp -o obj/cpp/organism.o
g++ -Wall  -DENABLE_CUDA -Werror  -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -O3 -c -g -gdwarf-3 -fPIC -std=c++11 -MMD src/neat.cpp -o obj/cpp/neat.o
nvcc -DENABLE_CUDA  -arch=sm_13 --debug --compiler-bindir gcc-4.8 -Xcompiler "-O3 -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -I/usr/local/cuda/include -L/usrlocal/cuda/lib64" -c src/network/cuda/cudanetwork.cu -o obj/cu/network/cuda/cudanetwork.o
nvcc -DENABLE_CUDA  -arch=sm_13 --debug --compiler-bindir gcc-4.8 -Xcompiler "-O3 -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -I/usr/local/cuda/include -L/usrlocal/cuda/lib64" -c -x cu src/experiments/maze/mazeevaluator.cxx -o obj/cu/cxx/experiments/maze/mazeevaluator.o
nvcc -DENABLE_CUDA  -arch=sm_13 --debug --compiler-bindir gcc-4.8 -Xcompiler "-O3 -Isrc -Isrc/experiments -Isrc/experiments/maze -Isrc/experiments/static -Isrc/innovgenome -Isrc/network -Isrc/network/cuda -Isrc/network/cpu -Isrc/species -Isrc/util -fopenmp -I/usr/local/cuda/include -L/usrlocal/cuda/lib64" -c -x cu src/experiments/static/staticevaluator.cxx -o obj/cu/cxx/experiments/static/staticevaluator.o
g++  obj/cpp/experiments/maze/maze.o obj/cpp/experiments/static/regex.o obj/cpp/experiments/static/cfg.o obj/cpp/experiments/static/xor.o obj/cpp/experiments/static/sequence.o obj/cpp/experiments/experiment.o obj/cpp/innovgenome/innovation.o obj/cpp/innovgenome/trait.o obj/cpp/innovgenome/innovnodegene.o obj/cpp/innovgenome/innovlinkgene.o obj/cpp/innovgenome/innovgenome.o obj/cpp/innovgenome/innovgenomemanager.o obj/cpp/genomemanager.o obj/cpp/population.o obj/cpp/network/cpu/cpunetwork.o obj/cpp/species/speciesorganism.o obj/cpp/species/species.o obj/cpp/species/speciespopulation.o obj/cpp/util/timer.o obj/cpp/util/rng.o obj/cpp/util/util.o obj/cpp/util/resource.o obj/cpp/util/map.o obj/cpp/main.o obj/cpp/organism.o obj/cpp/neat.o obj/cu/network/cuda/cudanetwork.o obj/cu/cxx/experiments/maze/mazeevaluator.o obj/cu/cxx/experiments/static/staticevaluator.o  -lgomp -lcudart -o neat
/usr/bin/x86_64-linux-gnu-ld: obj/cu/network/cuda/cudanetwork.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/maze/mazeevaluator.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/static/staticevaluator.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:49: recipe for target 'neat' failed
make: *** [neat] Error 1

EDIT 2: Makefile:

include Makefile.conf
CC_CUDA=nvcc -DENABLE_CUDA ${NVCC_FLAGS} -arch=sm_20 --debug --compiler-bindir ${PFM_NVCC_CCBIN} -Xcompiler "${OPT} ${INCLUDES} ${OPENMP} -I/usr/local/cuda/include -L/usr/local/cuda/lib64 "

INCLUDES=$(patsubst %,-I%,$(shell find src -type d))
SOURCES=$(shell find src -name "*.cpp")
CXX_SOURCES=$(shell find src -name "*.cxx")

OBJECTS=${SOURCES:src/%.cpp=obj/cpp/%.o}

LIBS=-lgomp
DEFINES=

ifeq (${ENABLE_CUDA}, true)
    CUDA_SOURCES=$(shell find src -name "*.cu")
    CUDA_OBJECTS=${CUDA_SOURCES:src/%.cu=obj/cu/%.o}
    CUDA_OBJECTS+=${CXX_SOURCES:src/%.cxx=obj/cu/cxx/%.o}
    LIBS+=-lcudart
    DEFINES+=-DENABLE_CUDA
else
    OBJECTS+=${CXX_SOURCES:src/%.cxx=obj/cpp/cxx/%.o}
endif

DEPENDS=${OBJECTS:%.o=%.d}
DEPENDS+=${CUDA_OBJECTS:%.o=%.d}

ifeq (${DEVMODE}, true)
    OPT=-O0
    #OPENMP=-fopenmp
    MISC_FLAGS=
    NVCC_FLAGS=-G -g
else
    OPT=-O3
    OPENMP=-fopenmp
    MISC_FLAGS=-Werror
    NVCC_FLAGS=--relocatable-device-code=true --dont-use-profile -ldir /usr/local/cuda/nvvm/libdevice 
endif

CC_FLAGS=-Wall ${DEFINES} ${MISC_FLAGS} ${PROFILE} ${INCLUDES} ${OPENMP} ${OPT} -c -g -gdwarf-3 -fPIC
.PHONY: clean default

default: ./neat

clean:
    rm -rf obj
    rm -f ./neat
    rm -f src/util/std.h.gch

./neat: ${OBJECTS} ${CUDA_OBJECTS}
    g++ ${PROFILE} ${OBJECTS} ${CUDA_OBJECTS} ${PFM_LD_FLAGS} -fPIC ${LIBS} -o $@

src/util/std.h.gch: src/util/std.h Makefile.conf Makefile
    g++ ${CC_FLAGS} -std=c++11 $< -o $@

ifeq (${ENABLE_CUDA}, true)

obj/cu/cxx/%.o: src/%.cxx Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    ${CC_CUDA} -c -x cu $< -o $@

obj/cu/cxx/%.d: src/%.cxx
    @mkdir -p $(dir $@)
    @${CC_CUDA} -M -x cu $< > $@.tmp
    @cat $@.tmp | sed 's,.*\.o[[:space:]]*:,$@:,g' | sed 's,\.d,\.o,' > $@
    @rm $@.tmp

obj/cu/%.o: src/%.cu Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    ${CC_CUDA} -c $< -o $@

obj/cu/%.d: src/%.cu
    @mkdir -p $(dir $@)
    @${CC_CUDA} -M $< > $@.tmp
    @cat $@.tmp | sed 's,.*\.o[[:space:]]*:,$@:,g' | sed 's,\.d,\.o,' > $@
    @rm $@.tmp

else
obj/cpp/cxx/%.o: src/%.cxx Makefile.conf Makefile
    @mkdir -p $(shell dirname $@)
    g++ ${CC_FLAGS} -std=c++98 -MMD $< -o $@
endif

obj/cpp/%.o: src/%.cpp Makefile.conf Makefile src/util/std.h.gch
    @mkdir -p $(shell dirname $@)
    g++ ${CC_FLAGS} -std=c++11 -MMD $< -o $@

-include ${DEPENDS}

Build log (just the end because of length limit):

g++  obj/cpp/experiments/maze/maze.o obj/cpp/experiments/static/regex.o obj/cpp/experiments/static/cfg.o obj/cpp/experiments/static/xor.o obj/cpp/experiments/static/sequence.o obj/cpp/experiments/experiment.o obj/cpp/innovgenome/innovation.o obj/cpp/innovgenome/trait.o obj/cpp/innovgenome/innovnodegene.o obj/cpp/innovgenome/innovlinkgene.o obj/cpp/innovgenome/innovgenome.o obj/cpp/innovgenome/innovgenomemanager.o obj/cpp/genomemanager.o obj/cpp/population.o obj/cpp/network/cpu/cpunetwork.o obj/cpp/species/speciesorganism.o obj/cpp/species/species.o obj/cpp/species/speciespopulation.o obj/cpp/util/timer.o obj/cpp/util/rng.o obj/cpp/util/util.o obj/cpp/util/resource.o obj/cpp/util/map.o obj/cpp/main.o obj/cpp/organism.o obj/cpp/neat.o obj/cu/network/cuda/cudanetwork.o obj/cu/cxx/experiments/maze/mazeevaluator.o obj/cu/cxx/experiments/static/staticevaluator.o  -fPIC -lgomp -lcudart -o neat
/usr/bin/x86_64-linux-gnu-ld: obj/cu/network/cuda/cudanetwork.o: relocation R_X86_64_32S against symbol `_ZTVN4NEAT11CudaNetworkE' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/maze/mazeevaluator.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: obj/cu/cxx/experiments/static/staticevaluator.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:50: recipe for target 'neat' failed
make: *** [neat] Error 1

EDIT 3: I was able to compile when I have added the options -fPIC -shared to -Xcompiler option and CC_FLAGS and ./neat: g++ line. When I run compiled binary, I get segmentation fault error. Additional info from valgrind program:

==14367== Jump to the invalid address stated on the next line
==14367==    at 0x1C296: ???
==14367==    by 0x12506B: operator<< <std::char_traits<char> > (ostream:561)
==14367==    by 0x12506B: cmp(NEAT::InnovationId const&, NEAT::InnovationId const&) [clone .part.5] (innovation.cpp:43)
==14367==  Address 0x1c296 is not stack'd, malloc'd or (recently) free'd
==14367== 
==14367== 
==14367== Process terminating with default action of signal 11 (SIGSEGV)
==14367==  Bad permissions for mapped region at address 0x1C296
==14367==    at 0x1C296: ???
==14367==    by 0x12506B: operator<< <std::char_traits<char> > (ostream:561)
==14367==    by 0x12506B: cmp(NEAT::InnovationId const&, NEAT::InnovationId const&) [clone .part.5] (innovation.cpp:43)
==14367== 

Answer 1

I tred a tip in this thread which suggested adding the following to the nvcc compiler options, which worked for me:

nvcc --compiler-options -fPIC ...

I only just learned that PIC stands for Position Independent Code (meaning that the executable can be loaded anywhere in memory). When compiling a PIC executable, all referenced libraries and objects need to be PIC as well (I think), including the Cuda code. So in my case the fPIC option needed to be passed to nvcc not g++.

Answer 2

Changing the setting of CC_FLAGS (the compilation options) from:

CC_FLAGS=-Wall ${DEFINES} ${MISC_FLAGS} ${PROFILE} ${INCLUDES} ${OPENMP} ${OPT} -c -g -gdwarf-3

to:

CC_FLAGS=-Wall ${DEFINES} ${MISC_FLAGS} ${PROFILE} ${INCLUDES} ${OPENMP} ${OPT} -c -g -gdwarf-3 -fPIC

seems likely to cure this particular linkage failure.