CODEWITHSUNDEEP
cqueuepthreadsmutex
Arjun Sethi
Arjun Sethi

Reputation: 36

What do you do if you enqueue faster than dequeue?

I am working with pthreads in C. I have thread 1 that receives data from a piece of hardware, and that data is placed into a queue. I have thread 2 that gets notified when data is in the queue and processes that data. Pseudo-code of my actual code is below.

#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>

void* inputThread(void *ptr){
    while(){
        getData(var1); //Notified by external hardware interrupt when data is available. var1 will hold new data
        enqueue(q1, var1);
        pthread_mutex_lock(&lock1);
            pthread_cond_signal(&condition1);
        pthread_mutex_unlock(&lock1);
    }
}

void* processDataThread(void *ptr){
    short data_to_process[];
    while(){
        pthread_mutex_lock(&lock1);
            while (emptyqueue(q1)) { 
                    pthread_cond_wait(&condition1,&lock1); //wait for inputthread to fill queue
                }
            data_to_process=dequeue(q1);
        pthread_mutex_unlock(&lock1);

        process_data(data_to_process); //causes thread to slow down
    }
}

int main(void){
    pthread_t input_thread_id;
    pthread_t thr[10];
    struct queue q1;
    struct queue q2;

    //
    //
    //some code to initiate queues, locks/mutex, conditions, etc.
    //
    //

    for (j) {
        pthread_create(&thr[j], NULL, processDataThread,(void*) &ptr);  
    }
    sleep(3); //sleep to let processthreads start up
    pthread_create(&input_thread_id, NULL, inputThread,(void*) &ptr);
}

I am sure that the enqueue and dequeue is working. I wrote out to a file and verified that whatever I receive in the inputThread is dequeued. I have a problem when the process_data() function is placed into the code. The function will slow down the processDataThread, causing the inputThread to fill the queue before processDataThread can dequeue. This results in missed data because the inputThread is working faster than the processDataThread. If I comment out the `process_data()' function, queue does not overflow. Even after increasing the number of processDataThreads (I have tried 3, 5, and 10), queue is still being enqueued faster than dequeued.

runtime getData() to dequeue() is .01s, and program continuously runs in loop without queue filling runtime getData() to process_data() is .05s, and queue fills up after a few seconds.

What am I doing wrong? I thought increasing the number of processDataThread would resolve the issue of enqueue faster than dequeue, but it did not.

Upvotes: 0

Views: 386

Answers (1)

ikegami
ikegami

Reputation: 385657

The function will slow down the processDataThread, causing the inputThread to fill the queue before processDataThread can dequeue.

That's normal. The work done by threads is often slower than the generation of the input for those threads (which could be as simple as reading lines from a file). Keep in mind that we use threads to parallelize slow processes.

This results in missed data

Something wrong with your queue if it's losing data!

I am sure that the enqueue and dequeue is working.

No, a queue that loses data isn't "working".

You didn't show the code you are trying to debug as required, so we can only guess at the problem. It sounds like you have a fixed-size queue and that enqueue just abandons the item to add to the queue if the queue is full. This is a bug.

If that's the issue, either grow the queue when it gets full, or block until there's space to enqueue what you want to enqueue. This working queue implementation does the latter.

Upvotes: 1

Related Questions