Make target before using it in a $(shell) call

Question

I have a project that can be configured via a scenario.xml file. The project contains a small tool to extract information from this xml file (tools/xmlq) which is compiled from a c source file. The Makefiles use this tool to influence the build by assigning its output to various make variables like

SOME_VAR := $(shell tools/xmlq some_query)

The problem with this approach is that on a clean build, tools/xmlq does not exist and SOME_VAR will not contain the expected value.

My question is: Is there some way to ensure that make builds tools/xmlq before the $(shell ...) calls that use it get executed? If not: what workarounds can I use to avoid the problem?

---

Some possible workarounds that I have thought of so far:

1. Use files instead of make variables.

   This means that a construct like

   SOME_VAR := $(shell tools/xmlq some_query)
   some-target: some-prerequisites
           some-command $(SOME_VAR) $^
   

   would change to

   some-var.txt: tools/xmlq scenario.xml
           $< some_query > $@
   some-target: some-var.txt some-prerequisites
           some-command $$(cat $<) $(filter-out $<,$^)
   

   this has the advantage that it feels very idiomatic but has the big disadvantage that I couldn't use e.g. $(SOME_VAR).o as a prerequisite any more.

2. Just call make tools/xmlq at the start of the makefile with $(shell make tools/xmlq)

   (There would obviously have to be some guards to avoid infinite recursion). The disadvantage here would be that the Makefiles are essentially read twice. Once to build tools/xmlq and once for the original invokation.

Answer 1

You don't require a workaround, just appropriate make logic. In detail this depends on where the source file xmlq.c resides. For illustration I'll assume it is in tools:-

SOME_VAR = $(shell tools/xmlq some_query)
some-target: some-prerequisites | tools/xmlq
        some-command $(SOME_VAR) $^

tools/xmlq: tools/xmlq.o
    $(CC) -o $@ $^ # Or however you build it.

There are two points to the solution:-

The immediate assignment

SOME_VAR := $(shell tools/xmlq some_query)

is replaced with the lazy assigment

SOME_VAR = $(shell tools/xmlq some_query)

So the definition $(shell tools/xmlq some_query) will only be expanded when $(SOME_VAR) is expanded, i.e. when the recipe some-command $(SOME_VAR) $^ is expanded, when make decides to run it. So tools/xmlq need not exist until that momment.

The rule

some-target: some-prerequisites | tools/xmlq

makes tools/xmlq an order-only prerequisite of some-target. Which means that tools/xmlq will not be considered in determining whether some-target should be made, but if it is determined that some-target should be made, then tools/xmlq will be made first. Thus tools/xmlq will exist before make expands $(SOME_VAR).

A demo:

tools/xmlq.c

#include <stdio.h>

int main(int argc, char *argv[])
{
    int i;
    for (i = 1; i < argc; ++i) {
        fputs(argv[i],stdout);
    }
    return 0;
}

and

Makefile

.PHONY: clean foo World

SOME_VAR = $(shell tools/xmlq Hello)
foo: World | tools/xmlq
    echo $(SOME_VAR) $^

tools/xmlq: tools/xmlq.o
    $(CC) -o $@ $^

World:;

clean:
    rm -fr tools/xmlq.o tools/xmlq

which runs like:

$ make
cc    -c -o tools/xmlq.o tools/xmlq.c
cc -o tools/xmlq tools/xmlq.o
echo Hello World
Hello World

And this might be reduced to:

.PHONY: clean foo World

foo: World | tools/xmlq
    echo $$(tools/xmlq Hello) $^

tools/xmlq: tools/xmlq.o
    $(CC) -o $@ $^

World:;

clean:
    rm -fr tools/xmlq.o tools/xmlq