Overriding malloc using the LD_PRELOAD and calls malloc in library functions

Question

I'm creating a small fast app which detects memory leaks in other apps. I use LD_PRELOAD for override default malloc and using this stored in the memory every call to malloc in your program. Problem is that some library functions of C use malloc too. Moreover, there are library functions which don't free allocated memory. Let's demonstrate:
MyApp.cpp

#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

static void * (* LT_MALLOC)(size_t) = 0;
static void (* LT_FREE)(void *) = 0;

static void init_malloc ()
{
  char *error;
  *(void **) (&LT_MALLOC) = dlsym (RTLD_NEXT, "malloc");
  dlerror ();
  if ((error = dlerror ()) != NULL)
    {
      fprintf (stderr, "%s\n", error);
      _exit(1);
    }
}

static void init_free ()
{
  char *error;
  *(void **) (&LT_FREE) = dlsym (RTLD_NEXT, "free");
  dlerror ();
  if ((error = dlerror ()) != NULL)
    {
      fprintf (stderr, "%s\n", error);
      _exit(1);
    }
}

extern "C"
void * malloc (size_t size) throw ()
{
  if (LT_MALLOC == 0)
    init_malloc ();
  printf ("malloc(%ld) ", size);
  void *p = LT_MALLOC(size);
  printf ("p = %p\n", p);
  return p;
}

extern "C"
void free (void *p) throw ()
{
  if (LT_FREE == 0)
    init_free ();
  printf ("free(%p)\n", p);
  LT_FREE(p);
}

And test.c. (Assume that a source code is not available initially, and I have a program only.)

#include <stdlib.h>
#include <stdio.h>
#include <time.h>

int main (void)
{
  printf ("test start\n");
  int *a = (int *)malloc(3 * sizeof (int));
  if (a)
    printf ("Allocated 3 int - %p\n", a);
  time_t t = time(NULL);
  a[0] = 1;
  printf ("time - %s", ctime(&t));
  printf ("a[0] = %d\n", a[0]);
  free (a);
  printf ("CALL FREE(a)\n");
  printf ("test done\n");
  return 0;
}

$ g++ -g -fPIC -c MyApp.cpp -o MyApp.o 
$ g++ -g -shared MyApp.o -o MyApp.so -ldl
$ gcc -g test.c -o test
$ export LD_PRELOAD=./MyApp.so

Run the program ./test and see:

malloc(72704) p = 0x8aa040
test start
malloc(12) p = 0x6a0c50
Allocated 3 int - 0x6a0c50
free((nil))
malloc(15) p = 0x6a0c70
malloc(552) p = 0x6a0c90
free((nil))
malloc(1014) p = 0x6a0ec0
free(0x6a0c90)
malloc(20) p = 0x6a0c90
malloc(20) p = 0x6a0cb0
malloc(20) p = 0x6a0cd0
malloc(21) p = 0x6a0cf0
malloc(20) p = 0x6a0d10
malloc(20) p = 0x6a0d30
malloc(20) p = 0x6a0d50
malloc(20) p = 0x6a0d70
malloc(21) p = 0x6a0d90
free((nil))
time - Mon Mar 14 18:30:14 2016
a[0] = 1
free(0x6a0c50)
CALL FREE(a)
test done

But I want to see:

test start
malloc(12) p = 0x6a0c50
Allocated 3 int - 0x6a0c50
time - Mon Mar 14 18:30:14 2016
a[0] = 1
free(0x6a0c50)
CALL FREE(a)
test done

I want my app avoid сalls malloc in library functions. But I no ideas how to do it.
Can malloc find out which function call it: library or your own? Or can we make to call default not override malloc in library functions? Or something else?
P.S. I am sorry for my bad English.

Answer 1

In the code posted, you do not track calloc, realloc and strdup. It would be more consistent to track all malloc related allocation functions.

You could also track fopen and fclose to disable malloc and free tracking during the execution of those. This would also allow you to track missing fclose() calls.

You can do the same for other C library functions that use malloc. It is unlikely that printf uses malloc, but you could track vfprintf to filter out any such calls. It is difficult to wrap variadic functions, but the printf family all call vfprintf or some close cousin, depending on your C library's implementation. Be careful to disable vfprintf tracking when you call printf yourself!

Answer 2

I have MANY times used the technique of compiling my code with a macro that replaces malloc with my own version. I thought I'd written an answer to demonstrate that, but apparently not. Something like this:

In "debugmalloc.h" or some such:

#if DEBUG_MALLOC
#define malloc(x) myTrackingMalloc(x, __FILE__, __LINE__)
#define free(x) myTrackngFree(x, __FILE__, __LINE__)

extern void* myTrackingMalloc(size_t size, const char* file, int line);
extern void myTrackingFree(void* ptr, const char* file, int line);
#endif

Then in a source file, e.g "debugmalloc.c":

#if DEBUG_MALLOC
void* myTrackingMalloc(size_t size, const char* file, int line)
{
    void *p = malloc(size);
    ... whatever extra stuff you need ... 
    return p;
}

void myTrackingFree(void* ptr, const char* file, int line)
{
    ... some extra code here ... 
    free(ptr);
}
#endif

[I allocated some extra bytes by modifying size and added then used an offset to return the appropriate actual payload pointer]

This is relatively easy to implement, and doesn't have the drawbacks of injecting using LD_PRELOAD - in particular that you don't need to distinguish between your code and libraries. And of course, you don't need to implement your own malloc.

It does have the slight drawback that it doens't cover new, delete, nor strdup and other library functions that in themselves do memory allocation - you can implement a new and delete global operator set [don't forget the array versions!], but replacing strdup and any other function that may allocate memory is quite a bit of work - and at that point, valgrind is probably going to be quicker, even if it runs quite slow.

Answer 3

Maybe the Dynamic Loader API can help you. There you find the function dladdr() which returns the module and the name of the symbol for a given address, if available.

Please see the man page of dlopen for a detailed explanation.