CODEWITHSUNDEEP
cmultithreading
Lopson
Lopson

Reputation: 1217

Pthreads - Increase maximum number of read locks

I have a multi-threaded C program implemented with pthreads that uses a read-write lock to protect a specific data structure. pthread_rwlock_rdlock, which is supposed to be a blocking call, can fail and return the value EAGAIN when invoked. The documentation says:

The pthread_rwlock_rdlock() and pthread_rwlock_tryrdlock() functions may fail if:

[EAGAIN]

The read lock could not be acquired because the maximum number of read locks for rwlock has been exceeded.

This means there's a maximum number of threads that can obtain the read lock at any given point in time. Taking this into consideration, I created a function that checks for the return value and loops endlessly until it actually gets the read lock.

void
cache_rdlock(void)
{
    int result= pthread_rwlock_rdlock(&cache_access);
    if(result== EAGAIN)
    {
        while((result= pthread_rwlock_rdlock(&cache_access))== EAGAIN);
    }

    return;
}

At a certain point during the program's execution, two concurrent threads trying to obtain this read lock will hang permanently in this function. Seeing as the program properly unlocks this read-write lock throughout its execution, what can I do to solve this problem? Is there a way to increase the maximum number of concurrent read locks? What changes should I make to this function in order for the program to work properly?

Upvotes: 3

Views: 1649

Answers (3)

Lopson
Lopson

Reputation: 1217

There was indeed a portion of the code where the lock wasn't being released properly, hence why it was hanging. Nonetheless, I've always had this question in my mind ever since I started using read-write locks, seeing as the documentation wasn't clear on where and how the maximum value was defined. Thanks for all the great and detailed input you've given. Hopefully, this question will serve as a warning and as a learning opportunity for other programmers out there!

Upvotes: 0

Maxim Egorushkin
Maxim Egorushkin

Reputation: 136218

You are probably forgetting to release the read lock.

Because given pthread_rwlock_t definition (x86_84):

typedef union
{
  struct
  {
    int __lock;
    unsigned int __nr_readers;
    unsigned int __readers_wakeup;
    unsigned int __writer_wakeup;
    unsigned int __nr_readers_queued;
    unsigned int __nr_writers_queued;
    int __writer;
    int __shared;
    unsigned long int __pad1;
    unsigned long int __pad2;
    /* FLAGS must stay at this position in the structure to maintain
       binary compatibility.  */
    unsigned int __flags;
# define __PTHREAD_RWLOCK_INT_FLAGS_SHARED  1
  } __data;

  char __size[__SIZEOF_PTHREAD_RWLOCK_T];
  long int __align;
} pthread_rwlock_t;

Overflow of unsigned int __nr_readers is most likely to occur if you keep forgetting to release the lock.

x86-64 implementation of pthread_rwlock_rdlock only ever returns EAGAIN on overflow of __nr_readers and __nr_readers_queued:

    /* Overflow.  */
8:  decl    NR_READERS(%rdi)
    movl    $EAGAIN, %edx
    jmp 9b

    /* Overflow.  */
4:  decl    READERS_QUEUED(%rdi)
    movl    $EAGAIN, %edx
    jmp 9b

Upvotes: 2

Art
Art

Reputation: 20392

Having implemented rwlocks I can quite confidently say that there's probably no way to increase the maximum number of concurrent read locks on your system and definitely no portable way of doing that.

At some basic level the rwlock contains some counter of the number of current read lock and that counter is a simple variable like an int or a short that's incremented for every read lock and decremented on unlock. If it's a short, you can yell at your operating system provider to make it bigger (even though it seems odd to hold 64k read locks) if it's an int, your program is probably broken and doesn't release the read locks because it should be pretty hard to obtain a billion or 4 read locks without a bug somewhere.

I'm saying a billion because a quite popular method to implement rwlocks is with a single 32 bit int that uses the two lowest bits for specifying a write lock.

Here's a simple test program you can use:

#include <pthread.h>
#include <stdio.h>
#include <limits.h>

int
main(int argc, char **argv)
{
    unsigned long long i;
    pthread_rwlock_t rw;
    int r;

    pthread_rwlock_init(&rw, NULL);


    for (i = 0; i < INT_MAX; i++) {
        if ((r = pthread_rwlock_rdlock(&rw)) != 0)
            break;
        if (i % 10000000 == 0)
            printf("%llu\n", i);
    }

    printf("%d %llu\n", r, i);

    return 0;
}

MacOS breaks at 16 million (2^24), Linux doesn't error out after 2 billion (2^31), so I didn't bother running it futher. You probably don't want to hold that many read locks.

Upvotes: 5

Related Questions