When to use recursive mutex?

Question

I understand recursive mutex allows mutex to be locked more than once without getting to a deadlock and should be unlocked the same number of times. But in what specific situations do you need to use a recursive mutex? I'm looking for design/code-level situations.

Answer 1

Seems no one mentioned it before, but code using recursive_mutex is way easier to debug, since its internal structure contains identifier of a thread holding it.

Answer 2

In general, like everyone here said, it's more about design. A recursive mutex is normally used in a recursive functions.

What others fail to tell you here is that there's actually almost no cost overhead in recursive mutexes.

In general, a simple mutex is a 32 bits key with bits 0-30 containing owner's thread id and bit 31 a flag saying if the mutex has waiters or not. It has a lock method which is a CAS atomic race to claim the mutex with a syscall in case of failure. The details are not important here. It looks like this:

class mutex {
public:
  void lock();
  void unlock();
protected:
  uint32_t key{}; //bits 0-30: thread_handle, bit 31: hasWaiters_flag
};

a recursive_mutex is normally implemented as:

class recursive_mutex : public mutex {
public:
  void lock() {
    uint32_t handle = current_thread_native_handle(); //obtained from TLS memory in most OS
    if ((key & 0x7FFFFFFF) == handle) { // Impossible to return true unless you own the mutex.
      uses++; // we own the mutex, just increase uses.
    } else {
      mutex::lock(); // we don't own the mutex, try to obtain it.
      uses = 1;
    }
  }

  void unlock() {
    // asserts for debug, we should own the mutex and uses > 0
    --uses;
    if (uses == 0) {
      mutex::unlock();
    }
  }
private:
  uint32_t uses{}; // no need to be atomic, can only be modified in exclusion and only interesting read is on exclusion.
};

As you see it's an entirely user space construct. (base mutex is not though, it MAY fall into a syscall if it fails to obtain the key in an atomic compare and swap on lock and it will do a syscall on unlock if the has_waitersFlag is on).

For a base mutex implementation: https://github.com/switchbrew/libnx/blob/master/nx/source/kernel/mutex.c

Answer 3

If you want to be able to call public methods from different threads inside other public methods of a class and many of these public methods change the state of the object, you should use a recursive mutex. In fact, I make it a habit of using by default a recursive mutex unless there is a good reason (e.g. special performance considerations) not to use it.

It leads to better interfaces, because you don't have to split your implementation among non-locked and locked parts and you are free to use your public methods with peace of mind inside all methods as well.

It leads also in my experience to interfaces that are easier to get right in terms of locking.

Answer 4

Recursive and non-recursive mutexes have different use cases. No mutex type can easily replace the other. Non-recursive mutexes have less overhead, and recursive mutexes have in some situations useful or even needed semantics and in other situations dangerous or even broken semantics. In most cases, someone can replace any strategy using recursive mutexes with a different safer and more efficient strategy based on the usage of non-recursive mutexes.

• If you just want to exclude other threads from using your mutex protected resource, then you could use any mutex type, but might want to use the non-recursive mutex because of its smaller overhead.
• If you want to call functions recursively, which lock the same mutex, then they either
  • have to use one recursive mutex, or
  • have to unlock and lock the same non-recursive mutex again and again (beware of concurrent threads!) (assuming this is semantically sound, it could still be a performance issue), or
  • have to somehow annotate which mutexes they already locked (simulating recursive ownership/mutexes).
• If you want to lock several mutex-protected objects from a set of such objects, where the sets could have been built by merging, you can choose
  • to use per object exactly one mutex, allowing more threads to work in parallel, or
  • to use per object one reference to any possibly shared recursive mutex, to lower the probability of failing to lock all mutexes together, or
  • to use per object one comparable reference to any possibly shared non-recursive mutex, circumventing the intent to lock multiple times.
• If you want to release a lock in a different thread than it has been locked, then you have to use non-recursive locks (or recursive locks which explicitly allow this instead of throwing exceptions).
• If you want to use synchronization variables, then you need to be able to explicitly unlock the mutex while waiting on any synchronization variable, so that the resource is allowed to be used in other threads. That is only sanely possible with non-recursive mutexes, because recursive mutexes could already have been locked by the caller of the current function.

Answer 5

If you want to see an example of code that uses recursive mutexes, look at the sources for "Electric Fence" for Linux/Unix. 'Twas one of the common Unix tools for finding "bounds checking" read/write overruns and underruns as well as using memory that has been freed, before Valgrind came along.

Just compile and link electric fence with sources (option -g with gcc/g++), and then link it with your software with the link option -lefence, and start stepping through the calls to malloc/free. http://elinux.org/Electric_Fence

Answer 6

I encountered the need for a recursive mutex today, and I think it's maybe the simplest example among the posted answers so far: This is a class that exposes two API functions, Process(...) and reset().

public void Process(...)
{
  acquire_mutex(mMutex);
  // Heavy processing
  ...
  reset();
  ...
  release_mutex(mMutex);
}

public void reset()
{
  acquire_mutex(mMutex);
  // Reset
  ...
  release_mutex(mMutex);
}

Both functions must not run concurrently because they modify internals of the class, so I wanted to use a mutex. Problem is, Process() calls reset() internally, and it would create a deadlock because mMutex is already acquired. Locking them with a recursive lock instead fixes the problem.

Answer 7

It would certainly be a problem if a thread blocked trying to acquire (again) a mutex it already owned...

Is there a reason to not permit a mutex to be acquired multiple times by the same thread?

Answer 8

For example when you have function that calls it recursively, and you want to get synchronized access to it:

void foo() {
   ... mutex_acquire();
   ... foo();
   ... mutex_release();
}

without a recursive mutex you would have to create an "entry point" function first, and this becomes cumbersome when you have a set of functions that are mutually recursive. Without recursive mutex:

void foo_entry() {
   mutex_acquire(); foo(); mutex_release(); }

void foo() { ... foo(); ... }