Is this a memory leak in Linux Mono application?

Question

I have been struggling to determine if what I am seeing is a memory leak in my Debian Wheezy Mono-Sgen 2.10.8.1-8 embedded application. The system has 512MB of RAM. Swap is disabled. I have been studying to try to understand how Linux manages process memory to conclude if what I am seeing is in fact a memory leak somewhere in Mono-Sgen. I am pretty sure it is not my application since I have profiled it numerous time after weeks of constant run time, and the GC memory always falls back to the baseline of the application. No objects are leaking from the viewpoint of my application. This does not mean internally Mono-Sgen is not leaking, but I have not determined this, and it may not be.

I have attempted to cap my Mono-Sgen heap since the default for mono for the large heap is 512MB, and since this is all my system has, I assumed I needed to cap it to prevent OOM from Linux. My configuration for Mono-Sgen is below:

export MONO_GC_PARAMS=major=marksweep-fixed,major-heap-size=32m,nursery-size=4m,evacuation-threshold=75

From my understanding I am using the mark-and-sweep fixed major heap with 32MB fixed size, default nursery size of 4MB and Mono-Sgen will perform a copying collection on the major heap if any of the major heap allocation buckets fall below being 75% utilized to prevent fragmentation. I bumped the default of 66% up.

My device has currently been up for a little over 6 days since a power outage caused a reset. When my application first starts up, I wait about 10 mins to make sure it is fully initialized, then I take a snap shot of the /proc/PID/status file to get a baseline of its memory usage. Today, I took another snap shop to see where I am, and as always, my virtual, resident, and data for this instance of the Mono-Sgen process has again grown. It has yet to breach the high water marks that occurred during initialization, but last time I did this test it did. What I have not been able to do and which I am trying to accomplish is to let it run to the point that it exhausts all of the physical memory of the system. I need to know if this is in fact a memory leak or if at some point Linux is going to reclaim some of the pages my process has allocated.

One thing I have notice is that even know I have no swap, the resident size of my Mono-Sgen process is always about 30MB less then the data count. From what I understand the data count is the amount of heap allocation, the resident size is what is actually in physical memory, the virtual is what has been allocated, not necessarily used.

My assumption is that Linux is just being Linux and not wasting time or memory unless it has to. I assume since the system is very lightly loaded, Linux has 0 memory pressure to do anything to reclaim memory and just lest Mono-Sgen keep on allocating and growing the heap, and my hope is when there is some actual memory pressure, Linux is going to step in and reclaim pages that are not really being used.

I have read that Linux will not shrink the allocated memory size of a process when free is called on previous allocated memory. I do not understand why, unless Linux being Linux, it will only do it when it has to. But my fear is how long I need to wait until this happens.

Is this a memory leak or will Linux reclaim the page difference seen between the resident and data size of this process when memory pressure starts to kick in? I have searched and read everything I can get my hands on about the subject and I am not finding the answer I am looking for and really do not want to wait a month to find out if my application is going to bounce because of OOM killer. I will anyway :) But I would like to know before hand.

I have looked into potential memory leaks with Mono-Sgen 2.10.8.1-8, but for what I am doing (using a lot of process.start() calls to native Linux applications) most of the type of bugs that would be hurting me are not in this release. I tried to update to Jessie's version of Mono-Sgen (3.2.8 I think) but it was crashing on my system so I reverted back to the stable Mono-Sgen 2.10.8.1-8 version out of more unknown fear.

Attached are numerous snap shots of the typical memory information, with most my attention being focused on what is returned by /proc/PID/status.

Any information will be greatly appreciated as always and my hope is I just do not understand how Linux reclaims memory on a lightly loaded system.

Answer 1

You can set following environment variables to control internal memory allocations by malloc. After setting it, it will answer you questions. If you want to know other options please refer :

http://man7.org/linux/man-pages/man3/mallopt.3.html

export MALLOC_MMAP_THRESHOLD_=8192

export MALLOC_ARENA_MAX=4