Reputation: 12673

Multi-node concurrency in Java

I've written a multi-threaded Java program to solve an embarrassingly parallel problem such that it utilizes all the free CPU cycles of on a multi-core CPU. I'd like to refactor my solution so that it can run on multiple nodes while still keeping the majority of the code I've already written.

I've used MPI with C in the past and been told that it's the "correct" way to address the issue of maximizing CPU cycles, but I'm also aware of other concurrent frameworks in Java like RMI and wonder if they are just as good.

Is there a good way to handle multi-node and multi-core concurrency in Java where the main goal is to leverage the most CPU cycles as possible out of the cluster?

Edit: I get the impression that there's no easy way to handle this stuff. I'm not surprised, but I was hoping. :)

Upvotes: 18

Answers (4)

Peter Lawrey

Reputation: 533870

You can try Hazelcast. It has a distributed ExecutorService. This should allow you to add tasks to a service which run across any number of nodes.

Upvotes: 7

oluies

Reputation: 17831

Depends on what you are doing and your budget you might want to look into (in no particular order)

Actors especially Akka that has good remote actors, STM and supervisor style managment with a Java API
Norbert
GridGain
Terracotta
Gigaspaces
Oracle Coherence
IBM Extreme Scale
TIBCO ActiveSpaces

Also see:

java util concurrent and guava (Presentation slides focusing on util.concurrent) (EventBus)
Libraries such as javolution or JSR 166
Functional programming capable JVM languages such as Scala and Clojure has better multi core utilization than Java.
RXJava (Java, Clojure, Scala ... Reactive Extentions)

Upvotes: 10

maerics

Reputation: 156642

Also consider Apache Hadoop, it uses MapReduce and is well suited for many parallel solutions.

Upvotes: 4

David Blevins

Reputation: 19378

JMS is a good place to start.

Upvotes: 5

Multi-node concurrency in Java

Answers (4)

Related Questions