How to continuously update baseline coverage file that's passed to genhtml?

Question

The latest version of genhtml provides a differential coverage report. ie it can tell you stats like if code in the previous version that was "covered" is now "uncovered" and vice versa. To make this work, you need to pass a baseline coverage file to genhtml that tells it the current state of coverage of your codebase. When someone adds a new commit to your codebase, genhtml uses the baseline file and the current state of coverage to give you differential stats like the one I mentioned above.

But once new changes have been added to the codebase, the baseline file is now stale and needs to be updated to the latest coverage state of the codebase. As you can see, you need to do this every time new code changes are made.

I don't want developers to be bothered by updating this baseline file every time they make new changes. I am thinking of automating updating of the baseline file via Github actions once every week or so. But wanted to see if someone has a better idea or does genhtml itself has a mechanism to avoid having to update the baseline file on every code change.

Answer 1

One way to do what I think you want is to use Jenkins (or your regression system of choice):

• define your "green" build as: passes all tests and passes your coverage criteria
• store the coverage data file(s) (.info) from your build in Jenkins 'Artifacts' (or somewhere you can find it again)
• when generating your new HTML report: use the stored coverage data from the 'most recent green build' as your genhtml --baseline-file parameter.
• use the gentml --criteria-script ... option to specify your coverage criteria (e.g., no un-exercised new code, no losses, etc.). genhtml will return a non-zero exit status if your criteria fails. See the man pages and the sample scripts, for details.
• Jenkins will now happily hum along without investigation and without intervention - as long as your developers continue to meet the criteria. With some additional scripting: you can further automate the system to send auto-mail to the likely guilty party if/when things fail.

Note that you almost certainly need a 'signoff' mechanism to deal with the (hopefully) rare case that you want to accept a commit when doesn't meet your criteria.

The above is called a 'coverage ratchet' in the paper referred in the LCOV readme.