CODEWITHSUNDEEP
makefile
Eric Wiener
Eric Wiener

Reputation: 5947

Automated testing with makefile

I'm extremely new to makefiles and just spent a full day trying to figure out how to automate my testing. For this project I have one program, main.c and it accepts as parameters an input and an output file. The input file is of the form "test-{filler}.txt" so an example file could be "test-invalid-opcode.txt". The output file would be of the form "out-{filler}.txt", so it would be of the form "out-invalid-opcode.txt". I would then want to compare the output with the correct output saved in "correct-{filler}.txt", so this would be "correct-invalid-opcode.txt". Each test would therefore have an input, output, and correct output. I would want to check if the output and correct output have any differences and I would want this to run on every test file with the prefix "test-". I read through a lot of the makefile spec and a lot of different examples, but I'm really confused as to how to handle this. Any help would be really appreciated.

Here is what I have to run one test:

CC=gcc
CFLAGS=-o

main.o: main.c
    $(CC) $(CFLAGS) main.o $<

.PHONY: test
test: main.o test-provided.txt test-out.cs
    ./program test-provided.txt test-out.txt
    diff -q test-out.txt out-provided.txt

The layout of what I want the automatic test to be is something like

.PHONY: autotest
autotest: program
    $(foreach <file with "test-" prefix> run:
        main.o <test-name> <out-name>
        diff -q <correct-name> <out-name>

But I really am lost on how to go about implementing that.

Thank you and sorry I couldn't post more code. I tried a lot of different things, but none of them were even close enough to be worth posting.

Edit to show working final version:

program: main.c
$(CC) $(CFLAGS) $a $<

TEST_INPUTS := $(wildcard test-*.txt)
.PHONY: autotest $(TEST_INPUTS)

autotest: $(TEST_INPUTS)

$(TEST_INPUTS): test-%.txt: program
    echo '' > out-$*.txt
    ./$< $@ out-$*.txt
    diff -q correct-$*.txt out-$*.txt

Upvotes: 0

Views: 1363

Answers (1)

MadScientist
MadScientist

Reputation: 100781

In general using loops with make is not very "make-ish". A makefile is an entire language fundamentally based on iteration and recursion so trying to do "extra" iteration inside a recipe is often redundant.

If you need to iterate over something, most especially when that something is files, you should attempt to work with make by taking advantage of its target/prerequisite organization. Above you give an algorithm:

$(foreach <file with "test-" prefix> run:
    main.o <test-name> <out-name>
    diff -q <correct-name> <out-name>

which is perfectly suited to make's default behavior. To translate this into a makefile you write a rule which invokes a single iteration of the loop, then use prerequisites to run it for the files you want. Something like this:

TEST_INPUTS := $(wildcard test-*.txt)
.PHONY: autotest $(TEST_INPUTS)

autotest: $(TEST_INPUTS)

$(TEST_INPUTS): test-%.txt: program
        $< $@ test-out.txt
        diff -q test-out.txt out-$*.txt

Not only is this much more make-like but it has other advantages: for example you can run make test-provided.txt and it will run just that one test instead of all the tests.

You have to use static pattern rules here not normal pattern rules because .PHONY targets can't work with pattern rules.

Upvotes: 1

Related Questions