CODEWITHSUNDEEP
gdbpthreadsmutexconditional-breakpoint
gremto
gremto

Reputation: 243

debugging pthreads with gdb

I'm trying to debug a very infrequent deadlock and I've narrowed it down to a problem with a pthread_mutex, which is of type 1 (recursive). I want to track down where this mutex is coming from and since all of our code uses normal mutexes, I thought it would make sense to detect when mutex type == recursive to trace it back.

I've tried setting a manual breakpoint in pthread_mutex_lock, dereferencing pthread_mutex_t via the stack pointer etc. to examine its type, but this is called millions of times and it would take forever to catch the case where mutex type == recursive.

I also tried interposing a library and replacing pthread_mutex_lock to make setting a breakpoint on the mutex type possible, but this didn't get any hits (not convinced this was catching all the calls to pthread_mutex_lock)

I get the feeling there must be a way in gdb of setting a watchpoint / conditional breakpoint for whenever pthread_mutex_lock is called with a mutex of type recursive?

Any help on the above would be greatly appreciated. Thanks in advance.

Upvotes: 0

Views: 3604

Answers (3)

Employed Russian
Employed Russian

Reputation: 213375

I've narrowed it down to a problem with a pthread_mutex, which is of type 1 (recursive) ...
I want to track down where this mutex is coming from and since all of our code uses normal mutexes

Presumably you've somehow determined that your thread(s) are blocked in pthread_mutex_lock trying to lock a recursive mutex, and you don't understand who is holding this mutex, and why.

The stack trace leading to pthread_mutex_lock should tell you exactly which code is trying to lock that mutex, which is all you should have to know to understand the problem.

I don't understand why you want to "catch" the pthread_mutex_lock in the act of locking that mutex, as that will likely not give you any more info than what you already have by looking at the stack after you've detected the deadlock.

In general, trying to debug mutex locking issues with GDB is futile -- the act of setting the breakpoints (or even just attaching GDB) changes timing of the operations to such an extent, that most issues never show up when running under GDB.

Upvotes: 1

Clement Bellot
Clement Bellot

Reputation: 841

You could try:

(gdb) conditional yourbreakpointid mutex.__m_kind==PTHREAD_MUTEX_RECURSIVE

Where mutex is the name of the mutex in the scope and yourbreakpointid the id of the breakpoint you placed in the function.

The __m_kind may change name depending on the implementation, search your distribution headers (pthread.h)if this one doesn't work.

Upvotes: 0

Stephane Rouberol
Stephane Rouberol

Reputation: 4384

Instead of gdb breakpoints, you can use watchpoints

Upvotes: 0

Related Questions