Compose a combined test-suite program from a collection of unit-tests

Question

I'm building a set of modules with unit-tests and I want to automatically generate a combined program to execute all the tests.

Indulging my inner control freak, I decided to use the minunit.h framework and try to do it all manually. So I have two source files (there will be more).

-rw-r--r-- 1 josh None 6192 Aug 21 23:15 io.c
-rw-r--r-- 1 josh None 2341 Aug 22 00:49 st.c

Each one has a unit-test embedded (so it's easier for me to keep updated, and maybe actually use it this time) guarded by #ifdef TESTMODULE. In outline:

... 
//module code 
... 

#ifdef TESTMODULE 
... 
//unit tests 
... 
int main(){ 
    // call unit tests 
} 
#endif 

I can then make a testing program with a very short file.

$ cat > io_test.c
#define TESTMODULE
#include "io.c"
$ make io_test
cc     io_test.c   -o io_test

I wrote a combined program by hand using the same sort of inclusion, but redefining any global names so they are unique.

# define main io_main 
# define tests_run io_tests_run 
# define all_tests io_all_tests 
# include "io_test.c" 
# undef main 
# undef tests_run 
# undef all_tests 
int io_test(){ 
    printf("running io_test\n"); 
    return io_main(); 
} 

# define main st_main 
# define tests_run st_tests_run 
# define all_tests st_all_tests 
# include "st_test.c" 
# undef main 
# undef tests_run 
# undef all_tests 
int st_test(){ 
    printf("running st_test\n"); 
    return st_main(); 
} 

int main(){ 
    return 
        0  || io_test()  || st_test()  ; 
} 

Now I want to generate this automatically from the much shorter list of variable parts: io and st. I can do most of the work with X-macros.

#define HASH #
#define UNITS(_) \
_(io) \
_(st) \
/**/

#define gen_unit_function(unit) \
HASH define main unit##_main \
HASH define tests_run unit##_tests_run \
HASH define all_tests unit##_all_tests \
HASH include STR(unit##_test.c) \
HASH undef main \
HASH undef tests_run \
HASH undef all_tests \
int unit##_test(){ \
    printf("running " STR(unit) "_test\n"); \
    return unit##_main(); \
}

#define gen_func_call(unit) \
    || unit##_test()

UNITS(gen_unit_function)
int main(){
    return
        0 UNITS(gen_func_call) ;
}

But of course, that can never work because you can't force a newline with the C preprocessor. What can I use to workaround this limitation?

Answer 1

Another option is to supplement the preprocessor with something like sed to chop the lines. Embed a token that sed can search for, like EOL.

#define HASH #
#define UNITS(_) \
_(io) \
_(st) \
/**/

#define gen_unit_function(unit) \
HASH define main unit##_main EOL\
HASH define tests_run unit##_tests_run EOL\
HASH define all_tests unit##_all_tests EOL\
HASH include STR(unit##_test.c) EOL\
HASH undef main EOL\
HASH undef tests_run EOL\
HASH undef all_tests EOL\
int io_test(){ EOL\
    printf("running " STR(unit) "_test\n"); EOL\
    return unit##_main(); EOL\
}

#define gen_func_call(unit) \
    || unit##_test()

UNITS(gen_unit_function)
int main(){
    return
        0 UNITS(gen_func_call) ;
}

And then pass it through sed after cpp.

cpp -P test_suite.inc | sed 's/EOL */
/g'

Answer 2

You can generate the source with m4, a much more powerful macro-processor.

Now m4 won't do X-macros the same way. You can't simply pass a macro name and reevaluate it in the expansion text. But the gnu docs describe (and distribute) a foreach macro which gives the same ability.

divert(`-1') 
# http://www.gnu.org/savannah-checkouts/gnu/m4/manual/m4-1.4.17/html_node/Foreach.html#Foreach 
# foreach(x, (item_1, item_2, ..., item_n), stmt) 
#   parenthesized list, simple version 
define(`foreach', `pushdef(`$1')_foreach($@)popdef(`$1')') 
define(`_arg1', `$1') 
define(`_foreach', `ifelse(`$2', `()', `', 
    `define(`$1', _arg1$2)$3`'$0(`$1', (shift$2), `$3')')') 

define(`UNITS', (io,st))

divert`'dnl 
`#' include <stdio.h> 
foreach(`unit', UNITS, ` 
`#' define main unit`'_main 
`#' define tests_run unit`'_tests_run 
`#' define all_tests unit`'_all_tests 
`#' include "unit`'_test.c" 
`#' undef main 
`#' undef tests_run 
`#' undef all_tests 
int unit`'_test(){ 
    printf("running unit`'_test\n"); 
    return unit`'_main(); 
} 
')dnl 

int main(){ 
    return 
        0 foreach(`unit', UNITS, ` || unit`'_test() ') ; 
} 

Then the program can be generated by running m4 all_tests.m4 > all_tests.c. And this can be added to the makefile.

all_tests.c:all_tests.m4 *_test.c
        m4 $< >$@

But going into this m4 file to update the list is going to be ugly, so take advantage of the naming convention and generate the list (io,st) with make (which knows how to examine filenames and directories and stuff).

$cat makefile
testprogs= $(notdir $(wildcard ./*_test.c)) 
unitprogs= $(subst _test,,$(testprogs)) 
units= $(basename $(unitprogs)) 

test:all_tests 
        ./all_tests 
all_tests.c:all_tests.m4 makefile $(unitprogs) 
        m4 -D UNITS="$(units)" $< >$@ 

Now since make likes space-delimited lists, but our m4 file is set-up to use a comma-delimited list, we'll need to pre-process the list before using it, producing this final version.

$cat all_tests.m4
divert(`-1') 
# http://www.gnu.org/savannah-checkouts/gnu/m4/manual/m4-1.4.17/html_node/Foreach.html#Foreach 
# foreach(x, (item_1, item_2, ..., item_n), stmt) 
#   parenthesized list, simple version 
define(`foreach', `pushdef(`$1')_foreach($@)popdef(`$1')') 
define(`_arg1', `$1') 
define(`_foreach', `ifelse(`$2', `()', `', 
    `define(`$1', _arg1$2)$3`'$0(`$1', (shift$2), `$3')')') 

define(`UNITS', (patsubst(UNITS,`\W',`,'))) 

divert`'dnl 
`#' include <stdio.h> 
foreach(`unit', UNITS, ` 
`#' define main unit`'_main 
`#' define tests_run unit`'_tests_run 
`#' define all_tests unit`'_all_tests 
`#' include "unit`'_test.c" 
`#' undef main 
`#' undef tests_run 
`#' undef all_tests 
int unit`'_test(){ 
    printf("running unit`'_test\n"); 
    return unit`'_main(); 
} 
')dnl 

int main(){ 
    return 
        0 foreach(`unit', UNITS, ` || unit`'_test() ') ; 
} 

Now whenever a new module is ready with a unit-testing main() guarded by TESTMODULE, it can be added to the suite by writing the same two-line file described in the question, fitting the pattern *_test.c.

#define TESTMODULE 
#include "zz.c"

And the unit-test and all_tests will all compile simply with make.

[This material was previously posted to comp.lang.c.]