How to run Valgrind in parallel with our process so its performance doesn't decrease too much?

Question

I need to use Valgrind to detect any memory access violations made in a server application. The server creates many threads. I suspect that there is a racing condition that causes the server to crash every 1 hour or so. We used Valgrind to analyze its memory usage but the server process' speed decreased dramatically. The server's speed decreased so much that it was hardly usable and no racing conditions where probable.

Is there anyway to run Valgrind in parallel with our application so we don't lose that much performance?

Answer 1

Valgrind works by hooking into your malloc calls, so you can expect your program to run slower under valgrind. So, I would say that you could not run your program faster under valgrind and get the benefit of analysing memory errors.

Answer 2

It's worth noting that Valgrind, while supporting multi-threaded programs, will not actually run the program's threads in parallel if you have multiiple cores available. It also interleaves threads at a finer grain than the native OS scheduler. These 2 facts combined might make it so a program with race conditions or other concurrent anomalies will behave differently.

You may want to try Helgrind, a tool primarily aimed at detecting correct locking discipline and drd, a tool primarily aimed at detecting data races.

Answer 3

You can't do that. Valgrind doesn't actually execute your code natively - instead it runs it inside a simulator. That's why it's so slow. So, there's no way to make it run faster, and still get the benefit of Valgrind.

Your best bet is to set the ulimit so that your program generates a core file when it crashes. Then you can try to work out what the problem was by examining the core.

Answer 4

This is not directly answering your question, but if you suspect a synchronization error, have you tried using the Valgrind tool Helgrind?