CODEWITHSUNDEEP
javamultithreadingqueue
yasar
yasar

Reputation: 13758

What type of queue to use to distribute jobs between worker threads

I am new to java and I want to write a threaded library as an exercise. It will work this way,

In the main thread, some jobs (as string) will be added to job queue, and when worker thread finishes the jobs, it will add it to finished queue. Main thread will get results from the finished queue. When all jobs are done, main thread will signal workers to stop. Here are some code I wrote so far:

public List<int> get() {
    WorkerThread[] threads = new WorkerThread[numThreads];
    LinkedList<int> results = new LinkedList<>();
    int workCount = 0;

    for (int i = 0; i < numThreads; i++) {
        threads[i] = new WorkerThread();
        threads[i].start();
    }
    // reader is a BufferedReader
    while ((String line = reader.readLine()) != null) {
        // put string to job queue
        workCount++
    }
    while(workCount) {
        //result = get result from finished queue, and add it to results LinkedList
        workCount--;  
    }
    for (int i = 0; i < numThreads; i++) {
        threads[i].canStop(); // this sets a private variable that makes infinite while loop to stop
        threads[i].join();
    }
    return results;
}

But I am confused with what kind of Queue implementation to use for this. As documentation shows 11 different kinds of Queue implementations.

Upvotes: 3

Views: 1701

Answers (3)

Ankur Lathi
Ankur Lathi

Reputation: 7836

A ConcurrentLinkedQueue is a unbounded thread-safe queue based on linked nodes.

It is an appropriate choice when many threads will share access to a common collection but this class does not permit the use of null elements.

This link contains a comparison between different queue implementation which can be used for multi-threading.

Upvotes: 1

le-doude
le-doude

Reputation: 3367

All the Queue's implementation in java.util.concurrent are advised if more than one thread access the same queue (actually they are all BlockingQueue implementations too). Even on read only functions (otherwise you might have the risk of two thread polling the same job from the queue and executing it). That is if you do not want to spend a great amount of time at synchronizing the accesses yourself (and testing the synchronization).

Then the choice depends on what extra functionality you want in your queue. The difference between LinkedBlockingQueue and ArrayBlockingQueue is in the performances of the various basic operations (insertion, deletion, selection, ...). PriorityBlockingQueue allows you to "push" (aka prioritize) some jobs compared to other depending on your own conditions (I believe it is implemented as a heap but do not quote me on that). DelayQueue does what the name says, it allows you to delay by a certain amount of time some jobs.

I like ConcurrentLinkedQueue since it uses a "wait-free" algorithm which in theories is better than synchronized (waiting for resource) functions.

On another note: I would advise you taking a look at Executor, Executors and ExecutorService rather than manage your own collection of threads.

Upvotes: 1

Joni
Joni

Reputation: 111349

Since the queue will be accessed from many different threads it'll probably be more efficient to use one of the queue implementations from java.util.concurrent. Other queues may require explicit synchronization.

Try ArrayBlockingQueue for example, it implements a bounded ring buffer. The fact that it's bounded means that it applies automatically back pressure when jobs are submitted faster than they can be processed; the queue doesn't just keep growing until you run out of memory.

The implementations differ mostly in performance characteristics, and different job loads may benefit from different queues. You should test which option is best for your situation.

Upvotes: 0

Related Questions