Ian Brockbank
Ian Brockbank

Reputation: 507

Why does clang show zero coverage for C++?

I am doing some fuzz testing using LLVM libFuzzer and Clang, but the coverage I am getting shows 0 hits for the whole file.

To simplify, I went back to the classic libFuzzer example (almost):

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
    if (size > 0 && data[0] == 'H')
    {
        if (size > 1 && data[1] == 'I')
        {
            if (size > 2 && data[2] == '!')
            {
                int *foo = (int *)44;
                *foo = 5;
            }
        }
    }
    return 0;
}

I compile it using clang -g -O1 -fsanitize=fuzzer -fprofile-instr-generate -fcoverage-mapping FuzzingApp.cpp -o FuzzingApp.exe

I run it: FuzzingApp.exe . It crashes as expected, and generates a default.profraw as expected.

I convert this to human-readable form:

llvm-profdata merge -sparse *.profraw -o default.profdata
llvm-cov show FuzzingApp.exe -instr-profile=default.profdata

The profile shows me 0 lines covered:

    1|       |// FuzzingApp.cpp : This file contains the 'main' function. Program execution begins and ends there.
    2|       |//
    3|       |
    4|       |#include <iostream>
    5|       |
    6|       |/**
    7|       | * @brief Fuzztest entry point.
    8|       | *
    9|       | * The LLVM fuzztest library will call this with random data and sizes, which
   10|       | * need to be interpreted as inputs to the ROM.
   11|       | *
   12|       | * @param   data        random data buffer of length size
   13|       | * @param   size        number of bytes of random data
   14|       | *
   15|       | * @return must be 0
   16|       | */
   17|       |extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
   18|      0|{
   19|      0|    if (size > 0 && data[0] == 'H')
   20|      0|    {
   21|      0|        if (size > 1 && data[1] == 'I')
   22|      0|        {
   23|      0|            if (size > 2 && data[2] == '!')
   24|      0|            {
   25|      0|                int *foo = (int *)44;
   26|      0|                *foo = 5;
   27|      0|            }
   28|      0|        }
   29|      0|    }
   30|      0|    return 0;
   31|      0|}

Since it crashed, it must have hit those lines. Why is it showing me it didn't?

Here's the version of clang I'm running (distributed with Visual Studio):

>clang --version
clang version 17.0.3
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\x64\bin

I have also tried it with the latest build of LLVM+Clang:

clang --version
clang version 18.1.8
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\clang\clang+llvm-18.1.8-x86_64-pc-windows-msvc\bin

Upvotes: 1

Views: 107

Answers (1)

Ian Brockbank
Ian Brockbank

Reputation: 507

As @MarekR alludes, there is no coverage output because the test crashes. If I run the same with few enough iterations that libFuzzer doesn't find the crash, coverage is generated.

> FuzzingApp -runs=10000
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 990453947
INFO: Loaded 1 modules   (8 inline 8-bit counters): 8 [00007FF7A7DB2008, 00007FF7A7DB2010),
INFO: Loaded 1 PC tables (8 PCs): 8 [00007FF7A7D816E8,00007FF7A7D81768),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2      INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 27Mb
#37     NEW    cov: 3 ft: 3 corp: 2/4b lim: 4 exec/s: 0 rss: 27Mb L: 3/3 MS: 5 CrossOver-ChangeByte-ChangeByte-ShuffleBytes-InsertByte-
#93     REDUCE cov: 3 ft: 3 corp: 2/3b lim: 4 exec/s: 0 rss: 27Mb L: 2/2 MS: 1 EraseBytes-
#103    REDUCE cov: 4 ft: 4 corp: 3/4b lim: 4 exec/s: 0 rss: 27Mb L: 1/2 MS: 5 CrossOver-CopyPart-EraseBytes-CrossOver-EraseBytes-
#3365   REDUCE cov: 5 ft: 5 corp: 4/6b lim: 33 exec/s: 0 rss: 27Mb L: 2/2 MS: 2 ChangeByte-ChangeBit-
#3436   NEW    cov: 6 ft: 6 corp: 5/9b lim: 33 exec/s: 0 rss: 27Mb L: 3/3 MS: 1 CrossOver-
#10000  DONE   cov: 6 ft: 6 corp: 5/9b lim: 98 exec/s: 0 rss: 27Mb
Done 10000 runs in 0 second(s)

> llvm-profdata merge -sparse *.profraw -o default.profdata
> llvm-cov show FuzzingApp.exe -instr-profile=default.profdata
    1|       |// FuzzingApp.cpp : This file contains the 'main' function. Program execution begins and ends there.
    2|       |//
    3|       |
    4|       |#include <iostream>
    5|       |
    6|       |/**
    7|       | * @brief Fuzztest entry point.
    8|       | *
    9|       | * The LLVM fuzztest library will call this with random data and sizes, which
   10|       | * need to be interpreted as inputs to the ROM.
   11|       | *
   12|       | * @param   data        random data buffer of length size
   13|       | * @param   size        number of bytes of random data
   14|       | *
   15|       | * @return must be 0
   16|       | */
   17|       |extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
   18|  10.0k|{
   19|  10.0k|    if (size > 0 && data[0] == 'H')
   20|  3.29k|    {
   21|  3.29k|        if (size > 1 && data[1] == 'I')
   22|    547|        {
   23|    547|            if (size > 2 && data[2] == '!')
   24|      0|            {
   25|      0|                int *foo = (int *)0;
   26|      0|                *foo = 5;
   27|      0|            }
   28|    547|        }
   29|  3.29k|    }
   30|  10.0k|    return 0;
   31|  10.0k|}

Thank you @MarekR

Upvotes: 0

Related Questions