non-recursive make with multiple architectures?

Question

I'm trying to improve our current GNU Make based build system. It is half non-recursive and half-recursive.

Following [1], it is non-recursive on the directory tree. While sources are organized across directories in logical modules, they're ultimately brought in to a single dependency graph.

However, it is also a multi-architecture build similar to what's described in [2] (such as 32-bit vs 64-bit, but also RELEASE vs DEBUG and Internal vs External, resulting in a large number of possible combinations), and that aspect is recursive: the top-level makefile calls itself with a given apps target with different variables (CPU_WIDTH=32 or =64).

This regularly causes problems, as certain targets need to be built only once at the top-level, while most others need to be built with appropriate flags per-architecture in the recursed levels. Regularly we discover that some build-once targets are actually being built multiple times, having been somehow swept into the recursive's dependency tree.

How could I design a Makefile such that it could contain the entire multi-architecture dependency graph in one non-recursive invocation of Make?

I feel like a key feature is using target-specific variables, which crucially propagate down the dependency graph. Unfortunately, a given named target will only be built once, even if it should be built multiple times with different options. A way around this would be to add some arch-specific info in the target name (for example, instead of target foo.o, make it 32/foo.o and 64/foo.o), but in the following naive example this didn't work:

.PHONY: all exe32 exe64 baz-$(ARCH)

all: exe32 exe64

exe32: ARCH = 32
exe32: baz-$(ARCH)
    @echo in $@ ARCH is $(ARCH)

exe64: ARCH = 64
exe64: baz-$(ARCH)
    @echo in $@ ARCH is $(ARCH)

baz-$(ARCH):
    @echo in $@ ARCH is $(ARCH)

This results in the erroneous output:

in baz- ARCH is 32
in exe32 ARCH is 32
in exe64 ARCH is 64

[1] Recursive Make Considered Harmful

[2] Multi-Architecture Builds

Answer 1

Ultimately, you need to duplicate your build targets for each architecture. Doing it by hand would be tedious, but there is an other way : making your entire build a template which can take the architecture as a parameter. And if you have other variable that affect the type of build (debug vs production, etc), then the entire arch-build also need to be a template which can be fed parameters. In order to do that, you need to keep in each of your templates a variable that will hold all of the preceding parameters, either as a prefix or a suffix on you target.

You can accomplish that with the use of call and eval:

## $1 will be the prefix holding parameters
define base-build

$1: $1-prog

.PHONY: $1

$1-prog: $(foreach dependency,$(dependencies),$1-dependency))
    $$(RECIPE)

$(foreach dependency, $(dependencies), $(call dependency-rule, $1, dependency))

endef

# $1 contains the prefix, $2 contains the list of arch
define arch-build
$(foreach $2,arch, $(call base-build,$1-$(arch))

$1: $(foreach arch,$2, $1-(arch))
.PHONY: $1

endef

# Same parameters use as arch-build
define build-type
$(foreach $2,type,$(call arch-build,$1-(type), $(ARCH_LIST))

$1: $(foreach type,$2, $1-(type))
.PHONY: $1

endef

TYPE_LIST := debug prod
ARCH_LIST := 32 64

$(eval $(call build-type,-,$(TYPE_LIST)))
all: -

.PHONY: all
.DEFAULT_GOAL:= all

The dependency-rule template should itself use the prefix in the target definition, except for non build-specific files such as sources files. And so on recursively.

You have to divide the set of your options which affect your build into subsets where each member are mutually exclusive (subsets would be arch, debug/prod etc). Then you shall decide you tree hierarchy (are arch target dependecy of debug/prod targets, or the other way around ).

I prefer to use a single eval since it's somewhat easier to debug (just replace it with info and you see what has not expanded as it should have. However, you should be careful to properly use empty lines in the templates definition ; if you aren't, they might be expanded without line break between them, and that will not work.

Doing it that way, your actual target names will look something like : debug-32-prog.

Recipes and target-specific variables

For recipes options which are specific to certain types of build, you can use target-specific variables.

First choice, you can set the target-specific variable in the top-level target in which it applies. I would do that by using computed variables names. You create another function for this :

define spec_flags # $1 = prefix, $2 arch (or other), $3 flag name.
$1-$2: $3: $($3_$2)

endef

(It can also be used for the other level in a similar manner). And then you add to your template for arch the following line

$(foreach arch,$2,$(call spec_flags,$1,arch,ARCH_FLAG)). This would set the values needed for each of the arch target, and all the prerequisites will inherit of these values.

But relying on that inheritance has a downside : you cannot directly specify a target lying lower in the dependency tree. Assuming you have file foo.c somewhere in your build, make prod-arch32-foo.o (for example) would bypass the prod and arch32 target, and hence not set the appropriate variables.

The other way is to propagate these variable at the same time than you build your dependency tree with your templates ; but i haven't found a simple way to do that yet.

Answer 2

The caveat (with re: to your case) of target specific variables appears as a consequence of the following sentence in the manual: "these values are only available within the context of a target’s recipe".

Recipes are evaluated in the second phase of make, after reading and evaluating all (or nearly all) variable substitutions, function calls and, most importantly: target names. Thus, at the time make creates a target from baz-$(ARCH): the variable $(ARCH) is not yet supplied with a value and therefore evaluates to the empty string, overall yielding the target baz-. There is no way to create a target from within another targets recipe process. Distinguishing between different configurations in make before the target generation process therefore requires you to set those variable in plain, non-target, non-rule lines of a makefile before their first use.

Reading your post again, I am a bit unsure if the problem you want to solve (as far as I understand), is to only have one rule for the double compilation step for target baz-$(ARCH):

.PHONY: all exe32 exe64 baz-$(ARCH)

all: exe32 exe64

exe32: ARCH = 32
exe32: baz-32
    @echo in $@ ARCH is $(ARCH)

exe64: ARCH = 64
exe64: baz-64
    @echo in $@ ARCH is $(ARCH)

baz-32:
    @echo in $@ ARCH is $(ARCH)

baz-64:   # this rule's code is a duplicate and you want to have only one copy
    @echo in $@ ARCH is $(ARCH)

The only way I know is to generate such rules by dynamic generation through a variable/function:

# create dynamic target; $1=target name, $2 = prerequisite list, $3 = recipe
define dyn-target =
.PHONY: $1
$1: $2 ;
    $3
endef

# instantiate all targets of given variants; $1=target name, $2 = prerequisite list, $3 = recipe, $4 = variant list
inst-targets = $(foreach var,$4,$(eval $(call dyn-target,$1-$(var),$(addsuffix -$(var),$2),$3))$(if $(make-debugout),$(info $(call dyn-target,$1-$(var),$(addsuffix -$(var),$2),$3))))

ALL-ARCHS = 32 64

all: exe32 exe64

exe32: ARCH := 32
exe64: ARCH := 64

exe32: foo-32 baz-32
exe64: foo-64 baz-64


define multiline-cmd =
    @echo in $$@
    @echo ARCH is $$(ARCH)
endef

$(call inst-targets,baz,,@echo in $$@ ARCH is +$$(ARCH)+,$(ALL-ARCHS))
$(call inst-targets,foo,,$(multiline-cmd),$(ALL-ARCHS))