Trying to view code coverage with llvm-cov

Question

I am building a library with a few *.c files and have a test file in subfolder test. The build tools are CMake, CLang and ninja. It runs on Windows 10 and on Unbuntu 16.04. I am trying to generate code coverage for viewing.

On Unbuntu my CMakeLists.txt include the line

set(CMAKE_CXX_FLAGS "-g -O0 -Wall -fprofile-arcs -ftest-coverage -fprofile-instr-generate -fcoverage-mapping -pthread")

I am not sure what these options do.

The compiler generates test.cpp.o and test.cpp.gcno. Running the test program generates test.cpp.gcda.

Running llvm-cov with these files causes

llvm-cov show: for the -instr-profile option: must be specified at least once!

I have also seen errors like

llvm-cov gcov: Not enough positional command line arguments specified!

I see USAGE: llvm-cov gcov [options] SOURCEFILE but have never seen an explanation or an example of what a SOURCEFILE is.

I am interested in seeing which lines of code are used at least once. What am I missing?

Answer 1

This is my Coverage.cmake file in SeeMake template:

function(EnableCoverage target)
    if(CMAKE_BUILD_TYPE STREQUAL Debug)
        if(UNIX)
            target_compile_options(${target} PRIVATE --coverage -fno-inline)
            target_link_options(${target} PUBLIC --coverage)
        elseif(WIN_CLANG)
            target_compile_options(${target}
                PRIVATE -fprofile-instr-generate -fcoverage-mapping -fno-inline)
            target_link_options(${target}
                PUBLIC -fprofile-instr-generate -fcoverage-mapping)
        endif()
    endif()
endfunction()

function(CleanCoverage target)
    add_custom_command(TARGET ${target} PRE_BUILD
    COMMAND "$<$<PLATFORM_ID:UNIX>:find ${CMAKE_BINARY_DIR} -type f -name '*.gcda' -exec rm {} +>"
            "<$<PLATFORM_ID:WIN32>:(ls -Path ${CMAKE_BINARY_DIR} -Filter *.profraw -Recurse).FullName | ForEach-Object -Process {del $_}>"
    )
endfunction()

function(AddCoverage target)
    if(UNIX)
        find_program(LCOV_PATH lcov REQUIRED)
        find_program(GENHTML_PATH genhtml REQUIRED)
        add_custom_target(coverage-${target}
            COMMAND ${LCOV_PATH} -d . --zerocounters
            COMMAND $<TARGET_FILE:${target}>
            COMMAND ${LCOV_PATH} -d . --capture -o coverage.info
            COMMAND ${LCOV_PATH} -r coverage.info '/usr/include/*'
                                 -o filtered.info --ignore-errors unused
            COMMAND ${GENHTML_PATH} -o coverage-${target}
                                    filtered.info --legend
            COMMAND rm -rf coverage.info filtered.info
            WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
        )
    elseif(WIN_CLANG)
        find_program(LLVM_COV_PATH llvm-cov REQUIRED)
        find_program(LLVM_PROFDATA_PATH llvm-profdata REQUIRED)
        add_custom_target(coverage-${target}
            COMMAND $<TARGET_FILE:${target}>
            COMMAND del coverage /S /Q
            COMMAND ${LLVM_PROFDATA_PATH} merge
                                    -sparse default.profraw -o default.profdata
            COMMAND ${LLVM_COV_PATH} show $<TARGET_FILE:${target}>
                                    -instr-profile=default.profdata
                                    -show-line-counts-or-regions
                                    -use-color
                                    -show-instantiation-summary
                                    -show-branches=count
                                    -format=html
                                    -output-dir=coverage-${target}
            COMMAND ${LLVM_COV_PATH} report $<TARGET_FILE:${target}>
                                    -instr-profile=default.profdata
                                    -show-region-summary=false
                                    -show-branch-summary=false
                                    -show-branch-summary=false
            WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
        )
    endif()
endfunction()

As you can see I have added the compile flags of -fprofile-instr-generate -fcoverage-mapping to the test target and run llvm-cov and llvm-profdata on the test executable.

Answer 2

Try running this command

llvm-cov gcov *.gcno

This will show you the coverage of all compiled source files.

Additional info about parameters:

-fprofile-arcs
Add code so that program flow arcs are instrumented. During execution the program records how many times each branch and call is executed and how many times it is taken or returns. On targets that support constructors with priority support, profiling properly handles constructors, destructors and C++ constructors (and destructors) of classes which are used as a type of a global variable.

When the compiled program exits it saves this data to a file called auxname.gcda for each source file. The data may be used for profile-directed optimizations (-fbranch-probabilities), or for test coverage analysis (-ftest-coverage). Each object file’s auxname is generated from the name of the output file, if explicitly specified and it is not the final executable, otherwise it is the basename of the source file. In both cases any suffix is removed (e.g. foo.gcda for input file dir/foo.c, or dir/foo.gcda for output file specified as -o dir/foo.o). See Cross-profiling.

--coverage
This option is used to compile and link code instrumented for coverage analysis. The option is a synonym for -fprofile-arcs -ftest-coverage (when compiling) and -lgcov (when linking). See the documentation for those options for more details.

Compile the source files with -fprofile-arcs plus optimization and code generation options. For test coverage analysis, use the additional -ftest-coverage option. You do not need to profile every source file in a program.
Compile the source files additionally with -fprofile-abs-path to create absolute path names in the .gcno files. This allows gcov to find the correct sources in projects where compilations occur with different working directories.
Link your object files with -lgcov or -fprofile-arcs (the latter implies the former).
Run the program on a representative workload to generate the arc profile information. This may be repeated any number of times. You can run concurrent instances of your program, and provided that the file system supports locking, the data files will be correctly updated. Unless a strict ISO C dialect option is in effect, fork calls are detected and correctly handled without double counting.
For profile-directed optimizations, compile the source files again with the same optimization and code generation options plus -fbranch-probabilities (see Options that Control Optimization).
For test coverage analysis, use gcov to produce human readable information from the .gcno and .gcda files. Refer to the gcov documentation for further information.
With -fprofile-arcs, for each function of your program GCC creates a program flow graph, then finds a spanning tree for the graph. Only arcs that are not on the spanning tree have to be instrumented: the compiler adds code to count the number of times that these arcs are executed. When an arc is the only exit or only entrance to a block, the instrumentation code can be added to the block; otherwise, a new basic block must be created to hold the instrumentation code.

-ftest-coverage
Produce a notes file that the gcov code-coverage utility (see gcov—a Test Coverage Program) can use to show program coverage. Each source file’s note file is called auxname.gcno. Refer to the -fprofile-arcs option above for a description of auxname and instructions on how to generate test coverage data. Coverage data matches the source files more closely if you do not optimize.