Determining if a memory leak is a false positive

Question

I have a simple program written in C++ that build in the following configurations:

1. Using/linked with libstdc++
2. Using/linked with libc++

I run both builds using valgrind like so:

valgrind --leak-check=full --show-reachable=yes --track-origins=yes --log-file=test_program.log -v ./test_program

The libstdc++ version runs and result in no memory leaks:

==
== HEAP SUMMARY:
==     in use at exit: 0 bytes in 0 blocks
==   total heap usage: 24,813,106 allocs, 24,813,106 frees, 51,325,970,073 bytes allocated
==
== All heap blocks were freed -- no leaks are possible
==
== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

However when libc++ is run it shows a memory leak:

==434036== HEAP SUMMARY:
==434036==     in use at exit: 16 bytes in 1 blocks
==434036==   total heap usage: 317,709,577 allocs, 317,709,576 frees, 645,827,127,171 bytes allocated
==434036==
==434036== Searching for pointers to 1 not-freed blocks
==434036== Checked 401,408 bytes
==434036==
==434036== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==434036==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==434036==    by 0x49A365F: ??? (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==434036==    by 0x49A24E9: __cxa_get_globals (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==434036==    by 0x49A53F6: __cxa_throw (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==434036==    by 0x2EE7B3: goal::details::special_node<double>::value() const (in workspace/goal/goal_test)
==434036==    by 0x2DC349: goal::details::caller_node<double>::value() const (in workspace/goal/goal_test)
==434036==    by 0x50AC40: double goal::details::arg_node<double>::process<main_node<double> > const&) (in workspace/goal/goal_test)
==434036==    by 0x45A79C: bool execute_test_base<double>() (in workspace/goal/goal_test)
==434036==    by 0x2322A0: main (in workspace/goal/goal_test)
==434036==
==434036== LEAK SUMMARY:
==434036==    definitely lost: 0 bytes in 0 blocks
==434036==    indirectly lost: 0 bytes in 0 blocks
==434036==      possibly lost: 0 bytes in 0 blocks
==434036==    still reachable: 16 bytes in 1 blocks
==434036==         suppressed: 0 bytes in 0 blocks
==434036==
==434036== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I have two further builds of the program:

1. Using/linked with libstdc++ with ASAN/LSAN/UBSAN/TSAN
2. Using/linked with libc++ with ASAN/LSAN/UBSAN/TSAN

When running them, neither of them trigger sanitizer errors or warnings.

Compilers used:

1. g++-13 (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0
2. clang version 20.0.0git

The valgrind leak is observed on both compilers only when linking with libc++.

---

Questions: Could the leak from valgrind be a false positive and what else can be done to verify it's legitimate?

Answer 1

Short answer
If you are using a very recent libc++ (or libc++ on Android or FreeBSD) then there will be no leak of any kind. The default changed recently with this commit:

commit acaa4c8bfd12eac4fa5f2c03e9d394f7bd8f1bd6
Author: Louis Dionne <ldionne.2@gmail.com>
Date:   Mon Jul 8 15:20:21 2024 -0500

    [libc++abi] Use __has_feature check to enable usage of thread_local for exception storage (#97591)
    
    Previously, we'd use HAS_THREAD_LOCAL which was never defined. Hence,
    we'd basically never use the code path where we use thread_local.
    
    Fixes #78207

The effect of this change is to switch to using TLS (no heap allocation). Without the change the default was to use a heap allocation and pthread_setspecific to store the pointer in a per-thread manner. There was NO DEALLOCATION.

Valgrind memcheck correctly reports the memory as still reachable.

The OP mentions clang 20.0.0git. Maybe not using that same version of libc++abi?

Long Answer
On platforms that enable thread_local for exception storage and with libc++ after the commit mentioned above you will get something like

==68104== Memcheck, a memory error detector
==68104== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==68104== Using Valgrind-3.24.0.RC1 and LibVEX; rerun with -h for copyright info
==68104== Command: ./test
==68104== 
--68104-- calloc(1,132) = 0x55DE040
--68104-- free(0x55DE040)
==68104== 
==68104== HEAP SUMMARY:
==68104==     in use at exit: 0 bytes in 0 blocks
==68104==   total heap usage: 1 allocs, 1 frees, 132 bytes allocated
==68104== 
==68104== All heap blocks were freed -- no leaks are possible

If you use GCC (and libstdc++) then things are slightly different. libstdc++ deliberately does not free up all reachable allocations. However it provides a function, __gnu_cxx::__freeres(), that Valgrind calls by wrapping exit(). You can see that if you run Valgrind with --run-cxx-freeres=no - 73728 bytes allocated from objlist_call_init still reachable.

On systems with libc++ 19.0 and earlier that don't explicitly enable thread_local exception storage you will get something like

./valgrind/vg-in-place --trace-malloc=yes --leak-check=full --show-reachable=yes ./test
...
--8572-- aligned_alloc(al 16, size 144) = 0x4CB8040
--8572-- calloc(1,16) = 0x4CB8110
--8572-- free(0x4CB8040)
...
==8752== HEAP SUMMARY:
==8752==     in use at exit: 16 bytes in 1 blocks
==8752==   total heap usage: 2 allocs, 1 frees, 160 bytes allocated
==8752== 
==8752== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==8752==    at 0x484BEFF: calloc (vg_replace_malloc.c:1675)
==8752==    by 0x49905B6: __cxxabiv1::__calloc_with_fallback(unsigned long, unsigned long) (llvm-project-18.1.8.src/libcxxabi/src/fallback_malloc.cpp:275)
==8752==    by 0x498F071: __cxa_get_globals (llvm-project-18.1.8.src/libcxxabi/src/cxa_exception_storage.cpp:81)
==8752==    by 0x49925EC: __cxa_throw (llvm-project-18.1.8.src/libcxxabi/src/cxa_exception.cpp:276)
==8752==    by 0x1091B1: main (so17.cpp:5)
==8752== 
==8752== LEAK SUMMARY:
==8752==    definitely lost: 0 bytes in 0 blocks
==8752==    indirectly lost: 0 bytes in 0 blocks
==8752==      possibly lost: 0 bytes in 0 blocks
==8752==    still reachable: 16 bytes in 1 blocks
==8752==         suppressed: 0 bytes in 0 blocks

EDIT: I fixed the trace for for aligned_alloc - see https://bugs.kde.org/show_bug.cgi?id=495469.

The flow in this case is

1. There's a aligned_alloc allocation from __cxa_allocate_exception. That will be memalign if you are using an older glibc.
2. There's a calloc allocation from __cxa_get_globals (see code in Patrick W.'s answer). There is no matching deallocation for this memory. The pointer returned by calloc gets stored in TLS with pthread_setspecific.
3. There's a call to free for the memory allocated in step 1.

If you build the code with AddressSanitizer or LeakSanitizer then it produces no errors. However if you run it with LSAN_OPTIONS=use_tls=0 then you get something like memcheck:

=================================================================
==1077==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x5dc8607484d9 in calloc (/home/paulf/testa+0x1184d9) (BuildId: cde96491b53a43657bbd4dc80eda7e27ef09c508)
    #1 0x75cb97ce15b6 in __cxxabiv1::__calloc_with_fallback(unsigned long, unsigned long) /usr/src/debug/libc++/llvm-project-18.1.8.src/libcxxabi/src/fallback_malloc.cpp:275:15
    #2 0x75cb97ce0071 in __cxa_get_globals /usr/src/debug/libc++/llvm-project-18.1.8.src/libcxxabi/src/cxa_exception_storage.cpp:81:17
    #3 0x75cb97ce35ec in __cxa_throw /usr/src/debug/libc++/llvm-project-18.1.8.src/libcxxabi/src/cxa_exception.cpp:276:31
    #4 0x5dc860793774 in main /home/paulf/so17.cpp:5:9
    #5 0x75cb979bbe07 in __libc_start_call_main /usr/src/debug/glibc/glibc/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #6 0x75cb979bbecb in __libc_start_main /usr/src/debug/glibc/glibc/csu/../csu/libc-start.c:360:3
    #7 0x5dc86065c0b4 in _start (/home/paulf/testa+0x2c0b4) (BuildId: cde96491b53a43657bbd4dc80eda7e27ef09c508)

I don't know LeakSanitizer very well. I see that by default it looks for blocks pointed to from TLS. "use_tls=0" causes LeakSanitizer to treat TLS like global memory instead. I can't see why the blocks that it finds pointed to from TLS don't get reported.

What can you do? Ideally use libc++ 19.1 or later where this is fixed.

In the meantime you could use a suppression.

Answer 2

Simple rule. Do not delude yourself into thinking that there may be false positives. It’s almost always just wishful thinking and confirmation bias.

Valgrind memcheck does generate some false positives but they are few and far between. Leak detection is an “easy” job and so the rate of false positives is essentially zero.

Memcheck can do a better job of leak detection than sanitizers because it has a complete view of the execution of the guest exe, from the very first instruction to the last. I’m not a sanitizer expert, but I imagine that the sanitizer code can only start when the first global constructor executes.

EDIT: the exe is catching the thrown exception. See my other answer for details.

Answer 3

The short of it: The situation you have described above is NOT a memory leak, in the sense that a piece of memory was allocated (eg: via malloc) assigned to a pointer, and later on the pointer was over-written or otherwise lost (scope), resulting in an explicit free of the allocated memory not being carried out.

So what is this leak report that valgrind is giving?

The Standard library implementations have a lot of leeway in regards to how they implement specified C++ standard features.

In your case, what you are seeing is that libc++, unlike libstdc++, creates a Thread Local Storage (TLS) instance of memory (via __calloc_with_fallback ) the first time the executable (aka process) attempts to throw an exception within a given thread (main is also considered as being a thread).

__cxa_eh_globals * __cxa_get_globals () {
//  Try to get the globals for this thread
    __cxa_eh_globals* retVal = __cxa_get_globals_fast ();

//  If this is the first time we've been asked for these globals, create them
    if ( NULL == retVal ) {
        retVal = static_cast<__cxa_eh_globals*>
                    (__calloc_with_fallback (1, sizeof (__cxa_eh_globals)));
        if ( NULL == retVal )
            abort_message("cannot allocate __cxa_eh_globals");
        if ( 0 != std::__libcpp_tls_set ( key_, retVal ) )
           abort_message("std::__libcpp_tls_set failure in __cxa_get_globals()");
       }
    return retVal;
    }

The code can be found here:

https://github.com/llvm/llvm-project/blob/main/libcxxabi/src/cxa_exception_storage.cpp#L80

The libc++ code that instantiates the memory immediately registers the associated pointer with a shutdown memory manager via the call to __libcpp_tls_set.

The idea here is that as part of the process shutdown procedure: Once all the state of the process has been free/destroyed/cleaned-up the TLS associated allocations (as there could be more than one thread in the process) registered with __libcpp_tls_set are then finally released (or freed).

Valgrind raises an issue here as it's not able to track the the true location of allocation and line it up with its associated destruction, though all the memory allocated since the start of the process running is eventually freed explicity - and not implicitly via OS clean-up.

The leak you are seeing can easily be replicated with the following code:

int main()
{
    try
    {
        throw 1;
    }
    catch (int)
    {} 

    return 0;
}

Build:

c++ -pedantic-errors -Wall -Wextra -Werror -O2 -o exceptiontest exceptiontest.cpp -L/usr/lib -lc++

Run valgrind:

valgrind --leak-check=full --show-reachable=yes --track-origins=yes --log-file=exceptiontest.log -v ./exceptiontest

Valgrind output:

==551373== HEAP SUMMARY:
==551373==     in use at exit: 16 bytes in 1 blocks
==551373==   total heap usage: 2 allocs, 1 frees, 160 bytes allocated
==551373==
==551373== Searching for pointers to 1 not-freed blocks
==551373== Checked 135,200 bytes
==551373==
==551373== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==551373==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==551373==    by 0x48AE65F: ??? (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==551373==    by 0x48AD4E9: __cxa_get_globals (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==551373==    by 0x48B03F6: __cxa_throw (in /usr/lib/llvm-14/lib/libc++abi.so.1.0)
==551373==    by 0x109105: main (in temp/exception_test/exceptiontest)
==551373==
==551373== LEAK SUMMARY:
==551373==    definitely lost: 0 bytes in 0 blocks
==551373==    indirectly lost: 0 bytes in 0 blocks
==551373==      possibly lost: 0 bytes in 0 blocks
==551373==    still reachable: 16 bytes in 1 blocks
==551373==         suppressed: 0 bytes in 0 blocks
==551373==
==551373== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

As you can see the written program is very simple and completely standards conforming and does not either explicitly or implicitly leak memory, but yet depending on the c++ standard library implementation used, valgrind will raise an issue related to non-freed blocks.

---

When building the program with libstdc++, as expected no leaks or reachable blocks are generated:

==551500== HEAP SUMMARY:
==551500==     in use at exit: 0 bytes in 0 blocks
==551500==   total heap usage: 2 allocs, 2 frees, 73,860 bytes allocated
==551500==
==551500== All heap blocks were freed -- no leaks are possible
==551500==
==551500== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)