CODEWITHSUNDEEP
c++boostshared-memory
Lagoda Alex
Lagoda Alex

Reputation: 11

boost interprocess vector does not deallocate shared memory

I am using boost::interprocess to share complex data structures between processes on linux. It mostly works, but my shared memory slowly leaks on each update of shared data.

After some debugging i was able to come up with relatively short way to reproduce this behavior:

#include <iostream>

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>


using UniversalAllocator = boost::interprocess::allocator
    < void
    , boost::interprocess::managed_shared_memory::segment_manager >;

using CharAllocator = boost::interprocess::allocator
    < char
    , boost::interprocess::managed_shared_memory::segment_manager >;

using ShmString = boost::interprocess::basic_string<char,   std::char_traits<char>, CharAllocator>;

struct Struct1
{
    Struct1(UniversalAllocator allocator)
        : member(allocator)
    {}
    struct Struct2
    {
        Struct2(UniversalAllocator allocator)
            : vector_member(allocator)
        {}
        struct Struct3
        {
            Struct3(UniversalAllocator allocator)
                : first(allocator)
                , second(allocator)
            {}
            ShmString first;
            ShmString second;
        };

        using Struct3Allocator = boost::interprocess::allocator
            < Struct3, boost::interprocess::managed_shared_memory::segment_manager >;
        using Structs3 = boost::interprocess::vector<Struct3, Struct3Allocator>;

        Structs3 vector_member;

    } member;
};

using Struct1Allocator = boost::interprocess::allocator
    < Struct1, boost::interprocess::managed_shared_memory::segment_manager >;

using Struct1Vector = boost::interprocess::vector
    < Struct1, Struct1Allocator>;


int main(void)
{
      boost::interprocess::managed_shared_memory segment(
          boost::interprocess::create_only,
          "TEST_MEMORY_LEAK",
          1500);
      std::cout << segment.get_free_memory() << std::endl;
      while(1)
      {
          Struct1Vector svector(segment.get_segment_manager());
          Struct1 selement(segment.get_segment_manager());
          svector.push_back(selement);
          std::cout << segment.get_free_memory() << std::endl;
      }
      return 0;
}

Memory is allocated inside the push_back call but only partly deallocated when leaving scope. As a result:

$ g++ --std=c++11 -Wall -pthread -c memory_leak.cpp -o memory_leak.o
$ g++ memory_leak.o -o memory_leak -pthread -lrt
$ ./memory_leak
1276
1180
1132
1084
1036
988
940
892
844
796
748
700
652
604
556
508
460
412
364
316
268
220
172
124
76
28
terminate called after throwing an instance of 'boost::interprocess::bad_alloc'
  what():  boost::interprocess::bad_alloc
Aborted

Strangely enought, if i replace shared memory size with 1000 (instead of 1500) the cycle is truely endless (it will end up printing 24 infinite number of times, so memory leaks to the last moment and then... stops?).

I am using boost 1.54 and gcc 4.8.4 if it matters, any help will be greatly appreciated, i somewhat running out of ideas :(

Upvotes: 1

Views: 564

Answers (2)

Lagoda Alex
Lagoda Alex

Reputation: 11

I went ahead and implemented my own memory management algorithm (as a thin wrapper over rbtree_best_fit which is default and therefore used in example above). I ended up seeing that code above allocates two segments on push_back - one of non-zero length and one of zero length and only deallocates the former one. That does not seem right to me so i searched boost website for an explanation and found what i believe is the issue. After upgrading system boost to 1.55 example above prints 1228 infinite number of times (presumably correct behavior).

Upvotes: 0

sehe
sehe

Reputation: 393769

Shared memory is not like a heap, but it's not your heap. Sharing implies locking and locking is expensive. What you'll notice is that the managed_buffer/managed_mapped_file/managed_shared_memory segment manager will reclaim memory as late as possible (possible only when an allocation fails).

If free memory overhead is too small at such a time, there might not be enough room to satisfy a contiguous allocation (so you'll preceive an allocation failure, as proof of a leak. In fact it's proof of fragmentation).

Upvotes: 0

Related Questions