GNU Make prerequisite rule not executed and dependent libs not found

Question

My problem is that a prerequisite (set by a variable) is not executed and even if executed manually before, required libraries are not found when linking. My Makefiles are non recursive, but the prerequisite is the external Software googletest and therefore called recursively, as the sources are not changed between several "make" runs.

My environment is:

$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ uname -a
Linux archlinux 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux

This is the relevant directory structure:

|-- ExampleTrunk
|   |-- BUILD
|   |   |-- BIN
|   |   |-- LIB
|   |   `-- OBJ
|   |-- buildenv
|   |   |-- ...
|   |   |-- GTEST.mk
|   |   |-- Makefile
|   |   |-- commons.mk
|   |   |-- module_commons.mk
|   |   |-- pathdefs.mk
|   |-- src
|       `-- HelloWorld
|           `-- module.mk
|   `-- test
|       `-- HelloWorld
|           `-- module.mk
`-- Tools
    `-- GoogleTest
        `-- googletest
            |-- make
                |-- Makefile

The folder ExampleTrunk/BUILD and its subdirectories are immediately created in pathdefs.mk and therefore exist before further rule execution.

The start Makefile is ExampleTrunk/buildenv/Makefile:

...
export SHELL := $(shell which bash)
...
.DEFAULT_GOAL := all
#some common function definitions
include commons.mk
#get $(PROJECT_ROOT), $(REL_PROJECT_ROOT); immediately create $(GLOBAL_OBJ_PATH), $(GLOBAL_LIB_PATH), $(GLOBAL_BIN_PATH)
include pathdefs.mk
export BUILD_DIRS := $(GLOBAL_BIN_PATH) $(GLOBAL_LIB_PATH) $(GLOBAL_OBJ_PATH)
include $(REL_PROJECT_ROOT)/src/HelloWorld/module.mk
...
#only include test stuff, when tests shall be built or cleaned to save dependency calculation time in all other cases
ifeq "$(call givenStringEndsWithTest,$(MAKECMDGOALS))" "0"
    include GTEST.mk
    include $(REL_PROJECT_ROOT)/test/HelloWorld/module.mk
endif

all : $(programs) | $(BUILD_DIRS);
all_Test: $(testPrograms) | $(BUILD_DIRS);
$(GLOBAL_OBJ_PATH)/%.o: %.cpp | $(BUILD_DIRS)
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@
clean:
    $(RM) $(addprefix $(GLOBAL_BIN_PATH)/,$(programs)) $(addprefix $(GLOBAL_BIN_PATH)/,$(testPrograms))
    $(RM) $(objectsToClean)
    $(RM) $(objectsToClean:.o=.d)
clean_Test: clean gtest_clean gmock_clean;

The rules "all", "clean" and "clean_Test" resolve and execute their prerequisites as expected and run without errors. (evaluated by make ... -p) The rule "all_Test" produces errors as described below.

GTest.mk contains the following recursive rules:

gtest: | $(BUILD_DIRS)
    [ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE)
gtest_clean:
    [ -d "$(REL_GTEST_MAKEFILE_DIR)" ] && cd "$(REL_GTEST_MAKEFILE_DIR)" && $(MAKE) clean

The directory "$(REL_GTEST_MAKEFILE_DIR)" exists, because it is only another path representation to the ExampleTrunk/BUILD/* directories that were immediately created before by pathdefs.mk. When I call any of both rules directly they can be executed without errors. Because I adapted the googletest Makefile, the outputs are created in the ExampleTrunk/BUILD/* directories as required.

The module.mk files are very short, as they include the module_commons.mk which contains most of the things to be executed:

ExampleTrunk/src/HelloWorld/module.mk:

additionalPrograms := 
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk

ExampleTrunk/test/HelloWorld/module.mk:

additionalPrograms := HelloWorld
gmockIsRequired := no
include $(PROJECT_ROOT)/buildenv/module_commons.mk

ExampleTrunk/buildenv/module_commons.mk generates most of the variables automatically from the module.mk file location and settings.

When I exectue

[.....ExampleTrunk/buildenv]$ make HelloWorld

everythink works fine, but with

[.....ExampleTrunk/buildenv]$ make HelloWorld_Test

it fails with a linker error (shown below). Here are the relevant *.mk parts for HelloWorld_Test with already resolved variables as shown from make HelloWorld_Test -p (variables with files/directories or flags are not written in resolved form to keep it short):

...
LDFLAGS_TEST := -L../BUILD/LIB -lgtest -lgtest_main
CPPFLAGS_TEST := $(googleStuffCPPFLAGS) # these are -I arguments for GCC, generated in GTEST.mk
moduleObj := "this variable is generated correctly by functions..."
...
HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);

../BUILD/BIN/HelloWorld_Test: $(moduleObj)
    g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $@

When I exectue

[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt

the log.txt shows that the rule variables are resolved correctly, but I get the error

/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status

This is because the rule "gtest" is not run before, but even if I run it manually by

[.....ExampleTrunk/buildenv]$ make gtest

and all required files are created in the correct ../BUILD/* directory, the error still remains.

What could be the reason, why 1. The prerequisite gtest is not executed for make HelloWorld_Test and 2. even if the libraries are created manually and therefore exist, they are not found by the linker

Thanks for your help Jasmin

Jasmin Smith · Accepted Answer

As stated in the comment before, the answer of MadScientist solved the first problem of building gtest before the test code itself, but linking still needed some changes to work. To get gtest to be built before the test sources I changed the build rules from:

HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test gtest | $(BUILD_DIRS);

../BUILD/BIN/HelloWorld_Test: $(moduleObj)
    g++ $(CPPFLAGS) $(CPPFLAGS_TEST) $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $@

to

HelloWorld_Test: ../BUILD/BIN/HelloWorld_Test;

../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
    g++ $(CXXFLAGS) $(LDFLAGS_TEST) $^ -o $@

The linking process doesn't need any header files. The required gtest libs were now built to ../BUILD/LIB before the *.o files are linked, but the libs still could not be found during the linking process:

[.....ExampleTrunk/buildenv]$ make HelloWorld_Test -p>log.txt

still gave the error

/usr/bin/ld: cannot find -lgtest
/usr/bin/ld: cannot find -lgtest_main
collect2: error: ld returned 1 exit status

The problem could be revealed by getting verbose output of the linking process by changing the rule to:

../BUILD/BIN/HelloWorld_Test: $(moduleObj) | gtest $(BUILD_DIRS)
    ld -o $@ $(LDFLAGS_TEST) $^ --verbose

The verbose output was:

GNU ld (GNU Binutils) 2.30
...
==================================================
attempt to open ../BUILD/LIB/libgtest.so failed
attempt to open ../BUILD/LIB/libgtest.a failed
...

so the problem was, that the parameter "-l..." adds the prefix "lib" to the library name. As the library names were gtest.a and gtest_main.a they could not be found. The solution was to change the variable from:

LDFLAGS_TEST := -L../BUILD/LIB/ -lgtest -lgtest_main

to the full path representation

LDFLAGS_TEST := ../BUILD/LIB/gtest.a ../BUILD/LIB/gtest_main

Now everything works fine :D

If more libs with "special" names have to be linked, it would be a better solution to change the corresponding Makefile to add the prefix "lib" to the library's output names. This would keep the command line shorter, as the path ../BUILD/LIB only appears once with the "-L" switch. Nevertheless I think for my case the full paths are okay, because the parameter length is not so much more and the manipulations to the 3rd party Makefile of googletest are less.

Thanks for your help, Jasmin

GNU Make prerequisite rule not executed and dependent libs not found

Answers (2)

Related Questions