CODEWITHSUNDEEP
cmemory-leaksvalgrindmemory-leak-detector
Pupkov-Zadnij
Pupkov-Zadnij

Reputation: 1392

how to find memory leaks in a very slow application?

I have a program, written in C, it uses Berkeley-Db, glib-2, and libxml-2. It is a search indexer which makes NLP with syntactic and semantic analysis, and stores conceptual structures in search indices instead of words.

However, I have a problem -- after a day or so, my program runs out of memory.

When I try to run my program under valgrind, it works very slow, it seems to me, that 10 times slower than with normal run.

All my attempts to check for memory-leaks (made for weeks under valgrind or so) didn't show any leaks.

Does anybody know some tools that can help me in this situation?

Upvotes: 3

Views: 575

Answers (1)

Peter Miehle
Peter Miehle

Reputation: 6070

my solution would be:

#define malloc(size) mymalloc(size, __FILE__, __FUNCTION__, __LINE__)
... (realloc, calloc, free)

void *mymalloc(size_t size, char *file, char *function, int line)
{
  void *data = malloc(size);
  addList(data, size, file, function, line);
  return data;
}

void myfree(void *data)
{
  removeList(data);
  free(data);
}

void debugalloc()
{
  printList();
}

the removeList() would search for "data" in the global list and remove it, or gives a warning. with "debugalloc();" you should print the whole list (and normally it should be empty, or well filled with what you expect it to have).

Another option: your memory management gets fragmented, ... (upps, Alexey Frunze wrote that just a second ago in the comments section.)

EDITH: XML

I made a comment about DOM and SAX, and I have the feeling just to expand this comment:

<head><data>...</data><data>...</data>......<data>...</data></head>

is an excellent candidate for SAX. In DOM, the whole XML will be read and converted into memory, which can lead very quickly to fragmented memory, if < data > itself is structured with different sized values. it may happen, that your operating system's memory modell will work against mallocing small amounts of memory.

With SAX, you only load portions of data into memory and get rid of it as soon as it is consumed. But your implementation must change from traversing the Document into a kind of State Machine with feeding (< data >) and consuming/processing (< / data >).

Upvotes: 2

Related Questions