CODEWITHSUNDEEP
javamultithreadingperformanceconcurrency
Ankit Kumar
Ankit Kumar

Reputation: 1463

Java: How to build a scalable Job processing mechanism

I need to build a job processing module wherein the incoming rate of jobs is in the order of millions. I have a multiprocessor machine to run these jobs on. In my current solution for Java, I use Java's ThreadPoolExecutor framework to create a job queue, a LinkedListBlockingQueue, and the number of threads equals the available processor on the system. This design is not able to sustain the incoming rate as the job queue keeps growing and within seconds it reports the overflow even though the CPU utilization is not maxed out. The CPU utilization remains somewhere in the range of 30-40 percent.

It means that most of the time is going away in thread contention where other CPU remains idle. Is there any better way of processing the jobs so that CPUs are utilized better so that job queue does not overflow?

Upvotes: 0

Views: 851

Answers (4)

Mak
Mak

Reputation: 616

Magical libraries are very tempting but often mislead you to wrong direction and makes your solution day by day more complex...Disruptor people LMAX says this too :) .. I think you should take a step back and understand the root cause of Job queue depth. In your case looks to me its same type of consumers so i don't think disruptor is going to help. You mentioned about thread contention. I would suggest first try to see if you can reduce the contention. Not sure if all your jobs are related but if not may be can use some portioning technique of queue and reduce unrelated jobs contention. Then you need to know why your consumers are slow. Can you improve your locking strategy by using ReadWrite locks or NonBlocking collections in consumers.

Upvotes: 0

Aaron Digulla
Aaron Digulla

Reputation: 328556

Your analysis is probably wrong. If the CPU was busy switching jobs, then the CPU utilization would be 100% - anything that the CPU does for a process counts.

My guess is that you have I/O where you could run more jobs. Try to run 4 or 8 times as many threads as you have CPU cores.

If that turns out to be too slow, use a framework like Akka which can process 10 million messages in 23 seconds without any special tuning.

If that's not enough, then look at Disruptor.

Upvotes: 1

David Billings
David Billings

Reputation: 376

Hi You could use a queuing system like RabbitMQ to hold messages for processing. If you combine this with Spring AMQP you can have easy (one line of config) multi-threading and the messages would be stored on disk until they are ready to be processed by your application.

Upvotes: 1

Peter Lawrey
Peter Lawrey

Reputation: 533442

I suggest you look at Disruptor first. This provides a high performance in memory ring buffer. This works best if you can slow the producer(s) if the consumers cannot keep up.

If you need a persisted or unbounded queue I suggest using Chronicle (which I wrote) This has the advantage that the producer is not slowed by the producer (and the queue is entirely off heap)

Both of these are designed to handle millions of messages per second.

Upvotes: 2

Related Questions