Any way to FORCE "dlclose()" to free memory?

Question

My application processes thousands of messages and uses dlopen / dlclose etc to call functions in shared libraries at runtime.

I have been analysing the memory at runtime and it seems (as I expected) that dlclose() does not free any malloc'ed memory after the close. So I have a rather bad memory leak....

The problem is these shared libs were written by someone else and I am not allowed to change the source. Is there any way around this? I suppose I could call a "subprocess" to process the message and then when it terminates the memory will dissappear for that subprocess....

Any other ideas?

Thanks for the help

Lynton

Answer 1

It's not clear from your question whether the memory "leaked" is memory allocated by the library you loaded, or by the dlopen implementation as part of managing the loaded libraries.

If it's allocated by the library, then either you are misusing the library and failing to call the right functions to instruct it to free memory it allocated, or it is poorly written and allocating (perhaps on first usage) data which it plans to keep around, reuse, and never free.

If it's allocated by the implementation of dlopen, then there's nothing wrong; this just means dlopen decided it can't safely unload the library, perhaps because there are still some references to its symbols hanging around. This is no problem since future loads of the same library will not consume more memory but will reuse the already-loaded copy.

Now, assuming the problem is not that you're misusing the library, all issues should be fixable simply by keeping the library open and not calling dlclose, and reusing it yourself if/when you need it next. That way, even if the library allocated memory behind your back and offers no way to free it, this will happen only once (the first time you load the library), not "N times", and thus it is not a "memory leak".

Answer 2

No, any memory allocated belongs to the process, not the library. In other words, dlclose has no way of knowing what memory was allocated by the library you're closing. Any well behaved library should provide a cleanup function, or never allocate memory that is not returned to you. Not all libraries are well behaved though...

Answer 3

I believe it is the memory allocated by the shared library which you are dlclose-ing which is staying, and you have no simple way to remove it (because you don't know which other parts of your process -e.g. which others dlopen-ed libraries is using it). If you want to understand more, read a good book on Garbage Collection, or at least the wikipage. Being a memory useful to the process is a global property of the entire process, not of particular libraries.

However, some libraries have conventions regarding memory usage, and might offer you the facility of cleaning up memory and resources. Others libraries don't release resources. Some libraries give you the ability to give as parameters the allocation routines they are calling.

You might consider using the Boehm conservative garbage collector or chase your leaks with an utility like valgrind.

Good luck, since your problem has no general solution. Perhaps telling us more about the actual libraries you are dlopen-ing might help.

And of course, there is the work-around of restarting from time to time your process.