How to analyze memory leak from coredump

Question

I would like to analyze a memory leak from core file analysis.

I have written sample code to inject memory leak and generate core file with gcore command.

#include <stdlib.h>
#include <unistd.h>
void fun()
{
  int *ptr = new int(1234);
}
int main()
{
  int i=0;
  while(i++<2500)
  {
    fun();
}
sleep(360);
return 0;
}

Find the process ID

ayadav@ajay-PC:~$ ps -aef |grep over  
ajay      8735  6016  0 12:57 pts/2    00:00:00 ./over  
ayadav    8739  4659  0 12:57 pts/10   00:00:00 grep over  

and generated core

ayadav@ajay-PC:~$ sudo gcore 8735
[sudo] password for ayadav:
0x00007fbb7dda99a0 in __nanosleep_nocancel () at ../sysdeps/unix/syscall-template.S:81
81      ../sysdeps/unix/syscall-template.S: No such file or directory.
Saved corefile core.8735

I have found common patterns from core file as below(as suggested on stackoverflow another thread Is there a way to locate which part of the process used the most of the memory, only looking at a generated core file?)

ayadav@ajay-PC:~$ hexdump core.6015 | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr | head
6913 0000000000000000  
2503 0000002100000000  
2501 000004d200000000  
786 0000000000007ffc  
464  
125 1ccbc4d000007ffc  
 92 1ca7ead000000000  
 91 0000000200007ffc  
 89 0000000100007ffc  
 80 0000000100000000  

Below two addresses are suspected one

2503 0000002100000000  
2501 000004d200000000  

core file have following repeated patterns

0003560 0000 0000 0021 0000 0000 0000 04d2 0000  
0003570 0000 0000 0000 0000 0000 0000 0000 0000  
0003580 0000 0000 0021 0000 0000 0000 04d2 0000  
0003590 0000 0000 0000 0000 0000 0000 0000 0000  
00035a0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035b0 0000 0000 0000 0000 0000 0000 0000 0000  
00035c0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035d0 0000 0000 0000 0000 0000 0000 0000 0000  
00035e0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035f0 0000 0000 0000 0000 0000 0000 0000 0000  
0003600 0000 0000 0021 0000 0000 0000 04d2 0000  
0003610 0000 0000 0000 0000 0000 0000 0000 0000  
0003620 0000 0000 0021 0000 0000 0000 04d2 0000  
0003630 0000 0000 0000 0000 0000 0000 0000 0000  
0003640 0000 0000 0021 0000 0000 0000 04d2 0000

But I do not have much idea how I can access it from command like gdb info address or x. Could anybody tell how I can convert symbol information from binary format?

Answer 1

1 - Memory leaks can be evaluated with a core dump. I have taken a sample c++ example:

class Base  
{  
public:  
    virtual void fun(){}  
    virtual void xyz(){}  
    virtual void lmv(){}  
    virtual void abc(){}  
};  

class Derived: public Base  
{  
public:  
    void fun(){}  
    void xyz(){}  
    void lmv(){}  
    void abc(){}  
};  

void fun()  
{  
    Base *obj  = new Derived();  
}  
int main()  
{  
    for(int i = 0; i < 2500;i++)
    {
        fun();
    }
    sleep(3600);
    return 0; 
}

2 - Created a core with gcore command

3 - Search the repeated pattern from core file.

ayadav@ajay-PC:~$ hexdump core.10639 | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr  | head
   6685 0000000000000000  
   2502 0000002100000000  
   2500 004008d000000000  
    726 0000000000007eff  
    502   
    125 2e4314d000007eff  
     93 006010d000000000  
     81 0000000100007eff  
     80 0000000100000000  
     73 0000000000000001  

0000002100000000 and 004008d000000000 are repeated patterns

4 - Check each qword is what with?

(gdb) info symbol ...
(gdb) x ...

example:

(gdb) info symbol 0x4008d000
No symbol matches 0x4008d000.
(gdb) info symbol 0x4008d0
vtable for Derived + 16 in section .rodata of /home/ayadav/virtual

5 - Probably most frequent vtable must relate to memory leak i.e Derived vtable.

Note: I agree coredump analysis is not best practice to find memory leak. Memory leak can be find with different static and dynamic tools like valgrind etc.

Answer 2

I don't think there is a way to identify whether a process is causing memory leak or not directly looking at the core dump. Infact, there is no thing called memory leak as such, we can not make that comment with out knowing the programmers intention for writing the code. having said that, you can get an idea by looking at the size of the core dump. You can generate multiple dumps say, one after initial run and one after prolonged run and if you see a vast difference in size, one can guess there could be something going wrong. But again, the memory could be used for productive purposes.

For the actual analysis and tracking of memory leak, one should be using tools like, memtrack, valgrind etc to add wrappers over malloc and free to give extra information about each alloc and free.

Update:

As you are looking for the hex analysis, here is what I can see: Your every line is 16 bytes and repeated in two lines. That is 32 bytes a chunk. 0x4D2 is 1234 in decimal. So, your data is there. It could be possible that your one alloc chunk is 32 bytes. Check and print the address in hex after every 'new()' and compare to see if you are observing a 32 bytes gap and then it explains it.