CODEWITHSUNDEEP
clinuxmultithreadinglinux-kernel
Frank Coool
Frank Coool

Reputation: 91

Linux kernel and multithreaded user application synchronization issue

I am having synchronization issues with my multithreaded user space application and simple kernel character driver. I am interested if I need synchronization mechanism in just user application or both user application and module.

Let me give you some overview:

I have 3 POSIX write threads that would open a char device (file) and on write system call, all three threads enqueue a global variable (static int token = 0 incremented after each write), take a second nap and try to enqueue again. The token is incremented in thread callback method (right after write operation) until the global variable is reached to 100. And similarly there are 3 read threads that would launch and call file read() system call and try to dequeue the token and print on the console (printk), take a nap and repeat the process until queue is empty.

I have a simple character driver that has open(), close() read() and write(). The driver has a Fifo queue implementation ( myqueue.c) and upon read() a variable is removed from the queue and copied to user space (value_remove_from_queue() and copy_to_user()) and upon write() variable is inserted to the queue (copy_from_user() and value_insert_to_queue()).

Pretty straightforward. However I do have some synchronization issues.

I tried to use mutex in userspace application. In thread callback that's where the token is incremented. example:

static int token = 1;
#define MAX_TOKEN 100
void write_callback(void* p)
{

   while( token <= MAX_TOKEN)
   {
      write(fd,buffer, 10);

      mutex_lock(&mymutex);
      token++;
      mutex_unlock(&mymutex);

      usleep(1);//1 second
   }
  }

Write() in character driver is similar to this:

static ssize_t syncdevice_write(struct file *f, const char __user *buf,
size_t len, loff_t *off)
{
   printk("(sync device) write()\n");
   if( pQueue == 0 )
   {  
      printk("(sync device) write() Queue does not exist.\n");
      return 0;
   }

   int size = RBuffer_Size(pQueue);
   printk("(sync device) write() Queue size = %d\n", size);

   long token = 0;
   char write_buffer[100];
   memset(write_buffer, 0, 100);

   if( copy_from_user(write_buffer, buf, strlen(buf)+1))
   {

     token  = atol( write_buffer);
     printk("(sync device) write long value = %ld\n", token);
     if(token)
     {
         RBuffer_Insert(pQueue, token);
     }
   }  
   return len;
}

My question is any synchronization in kernel code is necessary? Since I already have mutex lock in user space application. I do see unexpected results (for example) token not incremented properly (token value of 1 inserted multiple times).

Upvotes: 1

Views: 1683

Answers (1)

Tsyvarev
Tsyvarev

Reputation: 65898

General rule about mutex usage is that all accesses to the protected variable should be performed under mutex locked.

In your code access to token variable in token <= MAX_TOKEN condition checking is not protected by the mutex. Writing token into the buffer (for write() it) is not shown in your snippet, but it also should be performed inside critical section.

As for write() call itself, it is dependent of you purpose, whether it should be executed inside critical section. If you want precise order of tokens in the kernel's queue, then it should:

void write_callback(void* p)
{
    mutex_lock(&mymutex);
    while(token <= MAX_TOKEN)
    {
        int size = snprintf(buffer, sizeof(buffer), "%d", token);
        write(fd, buffer, size);
        token++;
        // Reaquire mutex for the next iteration. Also avoid sleeping with mutex held.
        mutex_unlock(&mymutex);
        usleep(1);
        mutex_lock(&mymutex);
    }
    mutex_unlock(&mymutex);
}

If you want only to observe all possible tokens(from 1 to 100) in the kernel queue, but order of them is not critical(e.g. it is acseptible to have 2,3,1,4 ...), you can move write() call outside of the critical section, e.g. place it immediately before usleep(1).

In any case, kernel's queue implementation should correctly process concurrent RBuffer_Insert() calls.

Upvotes: 1

Related Questions