CODEWITHSUNDEEP
makefile
Martin Ueding
Martin Ueding

Reputation: 8709

Makefile with chained rules runs last command needlessly on each run

I use TikZ to draw all kinds of vector graphics for my documents. It is rather slow and each drawing makes the compilation time of the main LaTeX document longer. Therefore I have all the figures as standalone files. They are compiled independently and the PDF file is included in the main document. This allows for parallel LaTeX runs on an as-needed basis.

The process looks as follows:

My complete makefile looks like this:

# Copyright © 2015-2016 Martin Ueding <[email protected]>

.PRECIOUS: %.tex %.pdf build/page/%.pdf

document_tex := $(wildcard physics*.tex)
document_pdf := $(document_tex:%.tex=%.pdf)

figures_tex := $(wildcard Figures/*.tex)
figures_pdf := $(figures_tex:Figures/%.tex=build/%.pdf)

all: $(figures_pdf)
#all: $(document_pdf)  # Disabled to only typeset figures right now.

test:
    @echo "document:    $(document_pdf)"
    @echo "figures_tex: $(figures_tex)"
    @echo "figures_pdf: $(figures_pdf)"

$(document_pdf): $(figures_pdf)

$(figures_pdf): build

build:
    mkdir -p build/page

build/page/%.tex: Figures/%.tex
    ../build-system/tikzpicture_wrap.py $< $@

build/%.pdf: build/page/%.pdf
    pdfcrop $< $@
    touch $@  # Added in an attempt to work around the problem, does not make any difference, though.

%.pdf: %.tex
    cd $$(dirname $@) && lualatex --halt-on-error $$(basename $<)

clean:
    $(RM) *-blx.bib
    $(RM) *.aux
    $(RM) *.log
    $(RM) *.run.xml
    $(RM) *.out
    $(RM) *.svg
    $(RM) *.pdf
    $(RM) -r build

It does work, it typesets all figures and they end up at build/*.pdf. The problem is that the pdfcrop step is run again and again, even when there is nothing more to do. In the output you can see the following:

pdfcrop build/page/propagator.pdf build/propagator.pdf
PDFCROP 1.38, 2012/11/02 - Copyright (c) 2002-2012 by Heiko Oberdiek.
==> 1 page written on `build/propagator.pdf'.
touch build/propagator.pdf

This is repeated for every single figure I have in my Figures directory.

I thought that this might be a problem with chained rules and added the intermediate file to the .PRECIOUS target for make to keep it. Now the files used by pdfcrop are not deleted midway.

Next I thought that it might be a problem with the timestamps on the files. If the source is newer than the target, make will run it. Therefore I added the touch to make sure that the target was newer than the source. This is not a problem as can be seen here after the run I did this morning. I did not change anything in that Figure since yesterday.

The source file. stat Figures/propagator.tex:

  File: 'Figures/propagator.tex'
  Size: 102             Blocks: 8          IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 17432618    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      mu)   Gid: ( 1000/      mu)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2016-01-16 10:58:34.384515470 +0100
Modify: 2016-01-16 10:58:34.369515566 +0100
Change: 2016-01-16 10:58:34.373515540 +0100
 Birth: -

The typeset PDF document. stat build/page/propagator.pdf:

  File: 'build/page/propagator.pdf'
  Size: 6265            Blocks: 16         IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 17432636    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      mu)   Gid: ( 1000/      mu)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2016-01-16 10:59:04.317323576 +0100
Modify: 2016-01-16 10:59:04.261323935 +0100
Change: 2016-01-16 10:59:04.261323935 +0100
 Birth: -

The cropped final file. stat build/propagator.pdf:

  File: 'build/propagator.pdf'
  Size: 6612            Blocks: 16         IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 17301550    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/      mu)   Gid: ( 1000/      mu)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2016-01-17 09:54:32.102511429 +0100
Modify: 2016-01-17 09:54:30.943517396 +0100
Change: 2016-01-17 09:54:30.943517396 +0100
 Birth: -

Still it performs all the pdfcrop operations again and again. I do not understand why. For the meantime I added the following kludge to make the compilation process finish a bit faster when there is nothing to do:

build/%.pdf: build/page/%.pdf
    if [ $< -nt $@ ]; then \
        pdfcrop $< $@; \
        fi

What is the actual problem here and how can I solve it?

Upvotes: 0

Views: 179

Answers (1)

MadScientist
MadScientist

Reputation: 100856

@Tsyvarev is on the right track. This is indeed the problem:

$(figures_pdf): build

However, it's not because build is not related to a particular file. build does exist: it's a directory which is created with the rule:

build:
        mkdir -p build/page

This builds the subdirectory build/page, which means that build exists (as a directory) and so in subsequent builds when make checks to see if it exists, it will determine that yes, it does. Make doesn't treat directories any different than any other file, when listed as a target.

However the existence of a prerequisite is not the only test that make uses: it also checks to see if the prerequisite is newer than the target. This is where directories and files behave differently, and why you virtually never (except in some special cases) want to use a directory as a normal prerequisite of a target.

A directory has a "time last modified" just like a file, and a directory's TLM value is updated just like a file's: when the directory is modified. What constitutes "modifying" a directory? Well, adding a new file, deleting an existing file, or renaming a file all modify the directory and cause it's TLM to change.

Probably you can see your problem now: every time your makefile adds, removes, or renames a file in the build directory its modification time is updated. This means that the directory build is always newer than almost all the files in it, which means they always rebuild every time.

There are two ways to work around this. One is to just always create the directory as a side-effect; remove build from the prerequisite list and remove the rule above, and instead simply force the directory to be created by make when it parses the makefile:

__dummy := $(shell mkdir -p build/page)

The second way, if you have a sufficiently new version of GNU make, is to use order-only prerequisites:

$(figures_pdf): | build

Upvotes: 2

Related Questions