CODEWITHSUNDEEP
javaquartz-schedulerdistributed-computingscheduler
SkyWalker
SkyWalker

Reputation: 14317

Open source Java Job Scheduler with: remoting, load balancing, failover, dependency DAG?

I'm looking for an open source Java Job Scheduler that allows sending different kind of jobs (not only flop intensive) and distributes them across many machines. It should also monitor the jobs and retry on different nodes should any job fail or the slave node crash. I would also appreciate load balancing similar to OpenMP or MPI. Ideally you should be able to pass in a Job dependency graph and the jobs will be processed in a topological-ordered fashion and parallelization should be done where possible.

The closest match I know to this is Quartz but this only allows scheduling single jobs by time and there is no remoting, failover, load balancing and dependency handling capabilities.

Such solution could be built on top of Quartz and a MOM server e.g. ActiveMQ but I'd like to be sure there is nothing out there first before building this up.

Probably a MapReduce port to Java would also do.

Upvotes: 1

Views: 1220

Answers (2)

Lolo
Lolo

Reputation: 4357

You should look at grid computing frameworks such as HTCondor, Hadoop (map/reduce), JPPF or GridGain, this is what they were made for.

Upvotes: 2

Shailesh Pratapwar
Shailesh Pratapwar

Reputation: 4224

Quartz do have a support for clustering . Check this.

Upvotes: 1

Related Questions