CODEWITHSUNDEEP
javamultithreadingqueue
PNS
PNS

Reputation: 19895

Multithreaded processing from Java queue(s)

Consider an application which uses an in-memory FIFO Java queue to deposit objects that will be subsequently processed by a thread, executing in parallel with many other threads (e.g., as part of a ThreadPool).

Each object must be processed by a "compatible" thread and that compatibility is assured via checking a label associated with the object (i.e., each thread can process certain types of objects, not all of them). If a thread reads an object from the queue and the label it reads is not among the ones it supports, it has to ignore the object.

Some additional characteristics the application has to fulfill:

  1. In-order processing (i.e., all objects with the same label should be processed in the order they are deposited in the queue).
  2. High performance (i.e., capable of processing thousands of rather "heavy" objects per second).

One could use the ConcurrentLinkedQueue, as suggested in an earlier question, but a single queue makes things tricky for segregating the input per label. Alternatively, each thread could be assigned with handling a single label, so it could have its own non-concurrent queue. Or maybe another approach should be followed.

What would be the best way to implement the above specification?

Upvotes: 0

Views: 1191

Answers (2)

Solomon Slow
Solomon Slow

Reputation: 27115

If a thread reads an object from the queue and the label it reads is not among the ones it supports, it has to ignore the object.

Does that mean that some of the tasks that go into the queue will never be performed? And does it mean that there's no way to predict which ones will be ignored and which ones will be processed?

That doesn't sound like a very good design.

If that's not what you meant, then maybe "ignored" was not the right word.

Each object must be processed by a "compatible" thread ... all objects with the same label should be processed in the order they are deposited in the queue

I'm not going to try to guess what could be different about the different kinds of worker threads, but If I was given that requirement, I would have a different thread pool for each different kind of worker. Then it becomes the responsibility of whoever generates the tasks to put them in the right queue.

Upvotes: 1

Manish Maheshwari
Manish Maheshwari

Reputation: 4134

Why would a thread reject a task based upon the object's label?

The thread that picks up the task from the queue must act like a worker thread. You must design/write in a manner, that the thread picks up the object (or task) from the queue, reads its label, and processes it based on its label.

You must not have a any correspondence between a object type and a thread type. Each thread should be written to handle any object.

Upvotes: 1

Related Questions