Partially compile and include boost with makefile (c++)

Question

I have a project in c++ that I want to be reasonably portable, and can be compiled by issuing a single "make" command. I currently use boost headers for certain operations, but now need "filesystem" from boost, which requires compilation.

I know there is a way to compile boost using the included shell script, but that takes FOREVER. I am curious of a way to selectively compile boost libraries, as specified from a makefile and include them in the linking process.

My current general idea is:

1. Have a makefile variable of the boost libraries wanted to be compiled and included.

2. Compile those libraries (before any dependent projects) from a boost distribution folder, AND output compiled binaries to a specific folder in the makefile project (eg: a "lib" folder)

3. I understand how to have these files included in the project by using:

   -Llib

What is the best way to do this, as well as choose where the boost output from compilation is? Should the files be .a or .o files? (or no extension?)

Answer 1

Using lockcmpxchg8b's answer, I was able to adapt it to my needs.

It was a little tricky to get makefiles to work right when recursively called, but I was able to get it to work.

SHELL = /bin/sh

# This makefile expects multiple arguments to be passed:
#
# Use the pattern: make var_name="var_value" when invoking this makefile
#
# BOOST_VER (the version suffix )
# BOOST_LIBS_TO_BUILD (space delimited list of boost libraries to build)
# BOOST_LIB_DIR (the output lib dir for the boost libraries)
# BOOSTDIR (the base directory to build from)
#

# Compile Info
CXX = g++
CXXFLAGS = -Wall -std=c++11

WORK_FOLDER = obj_boost$(BOOST_VER)

.PHONY: all
all: $(foreach lib,$(BOOST_LIBS_TO_BUILD),$(BOOST_LIB_DIR)/libboost_$(lib).a )

$(BOOST_LIB_DIR):
    @mkdir -p $(BOOST_LIB_DIR)

$(WORK_FOLDER):
    @mkdir -p $(WORK_FOLDER)

#####
# helper for building the .o files in WORK_FOLDER
#####
define MAKE_BOOST_LIB_COMPILE_RULES
$(foreach cppfile,$(shell ls $(BOOSTDIR)/libs/$(1)/src/*.cpp),$(WORK_FOLDER)/$(1)/$(notdir $(cppfile:.cpp=.o)): $(cppfile) | $(WORK_FOLDER)/$(1)
        $(CXX) $(CXXFLAGS) -D BOOST_ALL_NO_LIB \
                    -I$(BOOSTDIR) \
                    -c $$^ \
                    -o $$@
)
endef

#####
# define the build rules based on the files we find in the subfolders of
# the boost distro that correspond to our library names
#####
define BUILD_BOOST_LIB
$(WORK_FOLDER)/$(1): | $(WORK_FOLDER)
        @mkdir -p $$@
$(call MAKE_BOOST_LIB_COMPILE_RULES,$(1))
$(BOOST_LIB_DIR)/libboost_$(1).a: $(foreach cppfile,$(notdir $(shell ls $(BOOSTDIR)/libs/$(1)/src/*.cpp)),$(WORK_FOLDER)/$(1)/$(cppfile:.cpp=.o)) | $(BOOST_LIB_DIR)
        @ar r $$@ $$^
        @ranlib $$@
endef

#####
# dynamically generate the build rules from the list of libs
#####
$(foreach lib,$(BOOST_LIBS_TO_BUILD),$(eval $(call BUILD_BOOST_LIB,$(lib))))

.PHONY: clean
clean:
        @rm -rf $(WORK_FOLDER)
        @rm -rf $(BOOST_LIB_DIR)/*
        @echo "---- Done Cleaning Boost Libs ----"

To call this makefile, I call it from another makefile using the following:

# Boost
BOOST_VER= _1_65_1

BOOST_LIBS_TO_BUILD = filesystem timer chrono
#relative to this file
BOOST_LIB_DIR = shared/lib_boost$(BOOST_VER)
#relative to this file
BOOSTDIR = shared/boost$(BOOST_VER)

#Subdirectories
DIRECTORIES = $(sort $(dir $(wildcard */makefile)))

.PHONY: build
build: dependencies
        @$(foreach dir,$(DIRECTORIES),$(MAKE) -C $(dir);)

dependencies:
        @echo "---- Build Dependencies ----"
        @$(MAKE) -C shared -f build_boost_libs.mk BOOST_LIBS_TO_BUILD="$(BOOST_LIBS_TO_BUILD)" BOOST_LIB_DIR="../$(BOOST_LIB_DIR)" BOOSTDIR="../$(BOOSTDIR)" BOOST_VER="$(BOOST_VER)"

The pattern to CORRECTLY invoke a recursive make is:

$(MAKE) -C subdir_of_makefile -f makefile_name.mk

The use of $(MAKE) is the proper way to invoke make (as per documentation -- it ensures the same make command is used as the top-most make) and to correctly change the running directory, you need to use the -C argument (otherwise the directory context of any sub-make is the top-most parent makefile in the call stack). The -f option specifies a makefile that is anything but a default name for a makefile. This takes into account the -C option when looking for the named makefile.

I also found out that passing "arguments" to a makefile is a bit tricky, and is usually done with export and unexport, but the problem with those is due to ONLY the LAST "status" of a variable to be exported is used for the ENTIRE makefile. You can't export a variable, call a sub-make, then unexport. It would be unexported for the entire makefile run (as the last "status" of the variable was to be unexported despite it being called after the sub-make). Variable "status" is computed/parsed before the makefile executes.

Thanks for the help! I appreciate it. I'll also be releasing this makefile in an open-source project in the future (I'll credit your username as helping with that)

Answer 2

This is the wilder older brother of the prior answer's manual makefile. Given a path to the BOOST distro, BOOST_DISTRO, it finds the relevant source files for the target libs specified in TARGET_BOOST_LIBS, compiles them into $(WORK_FOLDER)/<lib>/, and archives the resulting objects into $(DEST_FOLDER)/libboost_<lib>.a.

BOOST_DISTRO=.
DEST_FOLDER=libs
WORK_FOLDER=build

TARGET_BOOST_LIBS=\
  system \
  filesystem \
  serialization

.PHONY: all
all: $(foreach lib,$(TARGET_BOOST_LIBS),$(DEST_FOLDER)/libboost_$(lib).a )

$(DEST_FOLDER):
    mkdir -p $(DEST_FOLDER)

$(WORK_FOLDER):
    mkdir -p $(WORK_FOLDER)

#####
# helper for building the .o files in WORK_FOLDER
#####
define MAKE_BOOST_LIB_COMPILE_RULES
$(foreach cppfile,$(shell ls $(BOOST_DISTRO)/boost/libs/$(1)/src/*.cpp),$(WORK_FOLDER)/$(1)/$(notdir $(cppfile:.cpp=.o)): $(cppfile) | $(WORK_FOLDER)/$(1)
    $(CXX) $(CXXFLAGS) -D BOOST_ALL_NO_LIB \
          -I$(BOOST_DISTRO)/boost \
          -c $$^ \
          -o $$@
)
endef


#####
# define the build rules based on the files we find in the subfolders of
# the boost distro that correspond to our library names
#####
define BUILD_BOOST_LIB
$(WORK_FOLDER)/$(1): | $(WORK_FOLDER)
    mkdir -p $$@
$(call MAKE_BOOST_LIB_COMPILE_RULES,$(1))
$(DEST_FOLDER)/libboost_$(1).a: $(foreach cppfile,$(notdir $(shell ls $(BOOST_DISTRO)/boost/libs/$(1)/src/*.cpp)),$(WORK_FOLDER)/$(1)/$(cppfile:.cpp=.o)) | $(DEST_FOLDER)
    ar r $$@ $$^
    ranlib $$@
endef


#####
# dynamically generate the build rules from the list of libs
#####
$(foreach lib,$(TARGET_BOOST_LIBS),$(eval $(call BUILD_BOOST_LIB,$(lib))))

.PHONY: clean
clean:
    -rm -rf $(WORK_FOLDER)
    -rm -rf $(DEST_FOLDER)

Testing with my ancient BOOST (#define BOOST_VERSION 105500), this builds the listed libs, and a dummy test program successfully compiles and calls boost::filesystem::absolute().