Understanding Makefile. make cannot link armadillo library

Question

I am new to C++ and I am having trouble understanding how Makefiles do their thing with the g++ compiler.

I have successfully installed armadillo library (via apt) and have a very simple c++ program test.cpp, like the one below:

#include <iostream>
#include <armadillo>

using namespace std;

int main()
{
   arma::mat A;

   A << -1 << 2 << arma::endr
       << 3 << 5;

   cout << A << endl;

   arma::fmat B;

   B.randu(4,5);

   cout << B;
   return 0;
}

This works just fine if I compile manually like this:

g++ src/test.cpp -std=c++11 -Wall -o test -DARMA_DONT_USE_WRAPPER -lopenblas -llapack

I can manually run the program and it delivers the matrices as expected.

On the other hand, I have the Makefile template from the VSCode C/C++ Extension, which I have modifed slightly for including the LAPACK an BLAS Fortran libraries:

########################################################################
####################### Makefile Template ##############################
########################################################################

# Compiler settings - Can be customized.
CC = g++
CXXFLAGS = -std=c++11 -Wall
LDFLAGS = -DARMA_DONT_USE_WRAPPER -lopenblas -llapack

# Makefile settings - Can be customized.
APPNAME = test
EXT = .cpp
SRCDIR = src
OBJDIR = obj

############## Do not change anything from here downwards! #############
SRC = $(wildcard $(SRCDIR)/*$(EXT))
OBJ = $(SRC:$(SRCDIR)/%$(EXT)=$(OBJDIR)/%.o)
DEP = $(OBJ:$(OBJDIR)/%.o=%.d)
# UNIX-based OS variables & settings
RM = rm
DELOBJ = $(OBJ)
# Windows OS variables & settings
DEL = del
EXE = .exe
WDELOBJ = $(SRC:$(SRCDIR)/%$(EXT)=$(OBJDIR)\\%.o)

########################################################################
####################### Targets beginning here #########################
########################################################################

all: $(APPNAME)

# Builds the app
$(APPNAME): $(OBJ)
    $(CC) $(CXXFLAGS) -o $@ $^ $(LDFLAGS)

# Creates the dependecy rules
%.d: $(SRCDIR)/%$(EXT)
    @$(CPP) $(CFLAGS) $< -MM -MT $(@:%.d=$(OBJDIR)/%.o) >$@

# Includes all .h files
-include $(DEP)

# Building rule for .o files and its .c/.cpp in combination with all .h
$(OBJDIR)/%.o: $(SRCDIR)/%$(EXT)
    $(CC) $(CXXFLAGS) -o $@ -c $<

################### Cleaning rules for Unix-based OS ###################
# Cleans complete project
.PHONY: clean
clean:
    $(RM) $(DELOBJ) $(DEP) $(APPNAME)

# Cleans only all files with the extension .d
.PHONY: cleandep
cleandep:
    $(RM) $(DEP)

#################### Cleaning rules for Windows OS #####################
# Cleans complete project
.PHONY: cleanw
cleanw:
    $(DEL) $(WDELOBJ) $(DEP) $(APPNAME)$(EXE)

# Cleans only all files with the extension .d
.PHONY: cleandepw
cleandepw:
    $(DEL) $(DEP)

I have passed the needed libraries under LDFLAGS = -DARMA_DONT_USE_WRAPPER -lopenblas -llapack. Nevertheless, this solution does not work. It looks to me like the compiler is unable to find the armadillo library, so I must have linked it somehow wrongly. It delivers:

g++ -std=c++11 -Wall -o test obj/test.o -DARMA_DONT_USE_WRAPPER -lopenblas -llapack
/usr/bin/ld: obj/test.o: in function `TLS wrapper function for arma::arma_rng_cxx11_instance':
test.cpp:(.text._ZTWN4arma23arma_rng_cxx11_instanceE[_ZTWN4arma23arma_rng_cxx11_instanceE]+0x25): undefined reference to `arma::arma_rng_cxx11_instance'
collect2: error: ld returned 1 exit status
make: *** [Makefile:36: test] Error 1

So, aside from the obvious question (Why does this not work?), I would as well appreciate if someone could help me clarify as well the following aspects:

On the one hand, rom the message error it seems that the command run g++ -std=c++11 -Wall -o test obj/test.o -DARMA_DONT_USE_WRAPPER -lopenblas -llapack does not include the name of the cpp file I wrote (as opposed to in my manual compilation, in which it works). Nevertheless, if I do not use armadillo, the Makefile recipe above works just fine. I see the Makefile somehow looking for all cpp files in the source code folder SRC = $(wildcard $(SRCDIR)/*$(EXT)), but I cannot see where is this forwarded to the compiler. Can someone help me with that?

The other thing is that, in my manual compilation, it seems to make no difference to pass the LAPACK and BLAS libraries as CXXFLAGS or LDFLAGS, meaning both of the following commands:

g++ src/test.cpp -std=c++11 -Wall -DARMA_DONT_USE_WRAPPER -lopenblas -llapack -o test

and

g++ src/test.cpp -std=c++11 -Wall -o test -DARMA_DONT_USE_WRAPPER -lopenblas -llapack

work just fine. As far as I have been able to read, I understood the flags before -o are meant for the compiler, and those after are meant for the "linker" (whatever that is). Can someone explain me what are the main differences between the CXXFLAGS and LDFLAGS? Why both combinations work? And what is the linker?

Thank you very much for your help.

Best,

D.

Answer 1

The other answer is a good general introduction to compilation but if you want to know what is happening in your situation you need to first understand that answer and the difference between source files, object files, and executable files and the way that they work, then go deeper to figure out what's wrong.

As far as I have been able to read, I understood the flags before -o are meant for the compiler, and those after are meant for the "linker" (whatever that is)

No, that is not right.

Turning source files into an executable involves several steps each managed by a different tool. The compiler front-end (e.g., g++) manages the order of these. Each of these may use different options, and whenever the compiler front-end invokes one of these tools it will pass the appropriate flags from the command line for that tool. It's not the case that "only" flags before or after -o are passed to different tools; it doesn't matter where on the command line they live.

The tools involved with compilation, in the order in which they're invoked, are:

1. Preprocessor: this handles #include and #ifdef and #define, etc. (the lines that start with # in your source). The preprocessor takes the options -D, -I, and some others.
2. Compiler: this turns your source code (after preprocessing to handle all the included files etc.) into assembly code which is very low-level: basically machine code but in ASCII form. This does the bulk of the work including optimization etc. Flags like -O2, -g, and many others are used by this tool.
3. Assembler: this turns the assembly code into a binary format for your CPU and generates an object file (foo.o).
4. Linker: this takes one or more object files plus libraries and turns them into an executable. This tool uses options like -L and -l to find libraries.

There's a separate tool, the archiver (ar) which is not invoked by the compiler front-end, which is used to turn object files (foo.o) into static libraries (libfoo.a).

Note, the above is a "classical" view of building: newer compilers munge the above steps together sometimes to get either better error messages or better optimization or both.

Most of the time the first three steps are all done by a single invocation of the compiler front-end: it turns a source file into an object file. You do this once for each source file. Then at the end, another invocation of the compiler front-end takes those object files and builds an executable.

If you look at the output make prints you'll see these two steps. First you'll see the compilation step, which is controlled by this make rule:

$(OBJDIR)/%.o: $(SRCDIR)/%$(EXT)
        $(CC) $(CXXFLAGS) -o $@ -c $<

and runs this command:

g++ -std=c++11 -Wall -o obj/test.o -c src/test.cpp

The -c option here tells the compiler, "do all the steps up to and including the compile step, then stop and don't do the link step".

Then you will see your link command, which is controlled by this make rule:

$(APPNAME): $(OBJ)
        $(CC) $(CXXFLAGS) -o $@ $^ $(LDFLAGS)

and runs this command:

g++ -std=c++11 -Wall -o test obj/test.o -DARMA_DONT_USE_WRAPPER -lopenblas -llapack

What do you notice about this? The -DARMA_DONT_USE_WRAPPER is a preprocessor option, but you're passing it to the link step and not passing it to the compile step. That means when the source is compiled, that option is not present and so whatever operation it was intended to suppress (using a wrapper apparently) is not being suppressed.

You need to put preprocessor options in a make variable that is sent to the compiler / preprocessor, so it should be this:

CXXFLAGS = -std=c++11 -Wall -DARMA_DONT_USE_WRAPPER
LDFLAGS = -lopenblas -llapack

Be sure to run clean before trying to build again.

Answer 2

One minor thing, but generally you should use CXX for your C++ compiler and CC for your C compiler (these are the usual conventions). If you do end up trying to compile C++ source with a C compiler you are likely to have problems. Less so the other way round.

So what it happening? Roughly speaking, you have two steps:

1. Compilation
2. Linking

When you compile a small exe, you can combine these into a single steps. Makefiles generally don't as two steps is more general.

For compilation the input has a .cpp suffix and you are passing the -c flag to tell the compiler to just compile. This will result in an object file (.o suffix).

For linking, there is no -c. The inputs are object files and the output is your application.

Other suffixes are possible (.cxx, .CC etc.).

There are 4 commonly used make variables

• CPPFLAGS for preprocessor flags, can be used for C and C++ compilation
• CFLAGS for flags specific to C compilation
• CXXFLAGS for flags specific to C++ compilation
• LDFLAGS for flags specific to linking

Historically, ld was the linker (and hence LDFLAGS), but it isn't smart enough to handle C++ linking well on its own. So now it is usually the C++ compiler that performs the task of "linker driver", that is g++ controls the linking that ld does.

Finally, your specific problem. You should add the armadillo library to LDFLAGS. The best way to do that is to just add -larmadillo. If armadillo is not installed in a 'standard' location like /usr/lib then you may need to additional arguments such as

-L/path//to/armadillo_lib -Wl,-rpath,/path//to/armadillo_lib

(the first one tells the linker where the library is, the second one puts that path into the executable so that is also knows where the library is).