CODEWITHSUNDEEP
makefilegnu-make
kyku
kyku

Reputation: 6042

Parallel makefile requires dependency ordering

I have the following piece of makefile:

CXXFLAGS = -std=c++0x -Wall
SRCS     = test1.cpp test2.cpp
OBJDIR   = object
OBJS     = $(SRCS:%.cpp=$(OBJDIR)/%.o)

all: test1 
release: clean test1

test1: $(OBJS)
    $(CXX) -o $@ $(OBJS)

$(OBJDIR)/%.o: %.cpp
    $(CXX) $(CXXFLAGS) -MD -c -o $@ $<

-include $(SRCS:.cpp=.d)

clean:
    rm -rf $(OBJDIR)/*

.PHONY: all clean release

Now if I try to invoke "make -j4 release" the clean target often gets execute in the middle of building files which causes compilation to fail. My question is how to ensure that the clean target has completed before starting the release build.

Upvotes: 32

Views: 24948

Answers (5)

theorifice
theorifice

Reputation: 690

I'm not exactly sure what versions this feature is supported in, but you can use the order-only feature:

my_target: dep1 dep2 | must_run_1st must_run_2nd

All dependencies left of the | character are processed as normal. Dependencies to the right of | are run 'order-only'

This feature is described at:

https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html

In your case, the following rules definition would suffice:

release: | clean test1
test1: | clean

Upvotes: 1

Eldar Abusalimov
Eldar Abusalimov

Reputation: 25493

You may split the execution into non-parallel (for release) and parallel (for the rest targets) phases.

ifneq ($(filter release,$(MAKECMDGOALS)),)
.NOTPARALLEL:
endif

release: clean
    $(MAKE) test1

.NOTPARALLEL target will suppress parallel execution if release target is mentioned in the command line. The release target itself will rerun Make after cleaning and build test1 in parallel.

UPD.

More clever solution would also reinvoke Make for each single target in case if there are more than one targets given on the command-line, so that a presence of release target would not force the rest to execute non-parallel too.

ifneq ($(words $(MAKECMDGOALS)),1)
.NOTPARALLEL:
$(sort all $(MAKECMDGOALS)):
    @$(MAKE) -f $(firstword $(MAKEFILE_LIST)) $@
else
# ...
endif

Update by James Johnston

The clever solution above doesn't work on versions of GNU make that don't support job servers. For example, released MinGW/native builds of GNU make prior to version 4.0 do not support job servers. (Cygwin/MSYS builds of GNU make do.) The code below uses the .FEATURES variable introduced in make 3.81 to detect if job servers are supported. The symptom of not using this workaround when it's needed is that your "parallel" build will be serialized.

# Check if job server supported:
ifeq ($(filter jobserver, $(.FEATURES)),)
# Job server not supported: sub-makes will only start one job unless
# you specify a higher number here.  Here we use a MS Windows environment
# variable specifying number of processors.
JOBSARG := -j $(NUMBER_OF_PROCESSORS)
else
# Job server is supported; let GNU Make work as normal.
JOBSARG :=
endif

# .FEATURES only works in GNU Make 3.81+.
# If GNU make is older, assume job server support.
ifneq ($(firstword $(sort 3.81 $(MAKE_VERSION))),3.81)
# If you are using GNU Make < 3.81 that does not support job servers, you
# might want to specify -jN parameter here instead.
JOBSARG :=
endif

ifneq ($(words $(MAKECMDGOALS)),1)
.NOTPARALLEL:
# The "all" target is required in the list,
# in case user invokes make with no targets.
$(sort all $(MAKECMDGOALS)):
    @$(MAKE) $(JOBSARG) -f $(firstword $(MAKEFILE_LIST)) $@
else

# Put remainder of your makefile here.

endif

Upvotes: 13

bobbogo
bobbogo

Reputation: 15483

In the release case, you need to ensure that clean completes before any compiling. Thus you (just) add it as a dependency to the compile rule (and not to the phony target). Several ways of doing this, like target-specific variables, or:

$(OBJDIR)/%.o: %.cpp $(if $(filter release,${MAKECMDGOALS}),clean)
    ...

Upvotes: 2

eriktous
eriktous

Reputation: 6649

For a solution without a recursive invocation of make, you could try this.

ifneq ($(filter release,$(MAKECMDGOALS)),)
test1: clean
endif

Upvotes: -1

Neil
Neil

Reputation: 55402

My preference is for

release:
    $(MAKE) clean
    $(MAKE) test1

This forces the two targets to be made consecutively without disturbing their inner parallelism (if any).

Upvotes: 39

Related Questions