CODEWITHSUNDEEP
c++cgcccode-coveragegcov
Dejan Jovanović
Dejan Jovanović

Reputation: 2125

Is it possible to check if code coverage has increased after a test

I am analysing coverage of a large project using the coverage support in gcc with gcov. I am also running random tests to try and increase the coverage. Ideally, as I generate a test, I would like to save it if it increases any coverage metric.

What would be a robust way to go about this?

For example, say we have something like

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char* argv[]) {
  int x, y;
  x = atoi(argv[1]);
  y = atoi(argv[2]);
  if (x < 0 || y < 0) {
    x = -x;
    y = -y;
  } 
}

Which I compile with 

gcc -fprofile-arcs -ftest-coverage -o test test.c

Is there a way to detect if the coverage has increased, and save the corresponding inputs? For example using something like 

while [ 1 ];
do
  ./gen_test.sh > input.txt
  ./test < input.txt
  if [ COVERAGE_INCREASED ] then save input.txt fi
done;

Running gcov to generate a report takes a lot of time, so I would prefer to know if the coverage has increased as soon as the program exits.

I know I can look at the .gcda files and see if they've changed. This, unfortunately, is not a solid solution as I am running the random tests in parallel and I would prefer that the "increase in coverage" test be atomic -- the project is large and number of gcda/source files is substantial.

I assume if something like this is possible, I will need to add some gcc-specific code to my main, but I have no idea where to look.

Attempt 1: I compiled the gcc from source (4.6.3) and looked into the gcov code. When the program under coverage test exits, it runs some merge functions to update the coverage data already in the files. I've updated one of these merge functions (libgcov.c) to just print something out if the coverage has been updated positively, which seems to serve my purposes.

/* The profile merging function that just adds the counters.  It is given
   an array COUNTERS of N_COUNTERS old counters and it reads the same number
   of counters from the gcov file.  */
void
__gcov_merge_add (gcov_type *counters, unsigned n_counters)
{
  gcov_type prev;
  for (; n_counters; counters++, n_counters--) {
    prev = gcov_read_counter ();
    if (prev == 0 && *counters > 0) {
      fprintf(stdout, "Coverage increase by add");
    }
    *counters += prev;
  }
}

It seems to be doing the right thing for line coverage, but it's missing the branch coverage. In addition, it doesn't capture the initial run but this is less of an issue.

Running the example above, I get

./test 0 0
./test -1 0
Coverage increase by add
./test 0 -1

Upvotes: 3

Views: 1346

Answers (2)

Dejan Jovanović
Dejan Jovanović

Reputation: 2125

I've solved it by overloading the add method of gcc in the code. For good practice, in order to obtain gcov_type you can install the gcc-4.6-plugin-dev package.

Now, the above program can be adapted as 

#include <stdio.h> 
#include <stdlib.h>

#include <gcc-plugin.h>
#include <basic-block.h>

gcov_type __gcov_read_counter (void);

void
__gcov_merge_add (gcov_type *counters, unsigned n_counters)
{
  gcov_type prev;
  for (; n_counters; counters++, n_counters--) {
    prev = __gcov_read_counter ();
    if (prev == 0 && *counters > 0) {
      fprintf(stdout, "Coverage increased\n");
    }
    *counters += prev;
  }
}

int main(int argc, char* argv[]) {
  int x, y;
  x = atoi(argv[1]);
  y = atoi(argv[2]);
  if (x < 0 || y < 0) {
    x = -x;
    y = -y;
  } 
}

Compiling above need some extra includes

gcc -I/usr/lib/gcc/x86_64-linux-gnu/4.6/plugin/include \
    -fprofile-arcs -ftest-coverage \
    -o test test.c

This binary will print prints "Coverage increased" each time coverage improves.

Upvotes: 1

dascandy
dascandy

Reputation: 7292

It is possible to read the gcov structs from your running program while it runs / before it exits. Will add details if you're interested tomorrow, on a phone now.

Upvotes: 0

Related Questions