CODEWITHSUNDEEP
javap2pdistributed-computingmesosjxta
Tom
Tom

Reputation: 330

Should I be using a distributed system like Mesos?

I have a project which briefly is as follows: Create an application that can accept tasks written in Java that perform some kind of computation and run the tasks on multiple machines* (the tasks are separate and have no dependency on one another).

*the machines could be running different OSs (mainly Windows and Ubuntu)

My question is, should I be using a distributed system like Apache Mesos for this?

The first thing I looked into was Java P2P libraries/ frameworks and the only one I could find was JXTA (https://jxta.kenai.com/) which has been abandoned by Oracle. Then I looked into Apache Mesos (http://mesos.apache.org/) which seems to me like a good fit, an underlying system that can run on multiple machines that allows it to share resources while processing tasks. I have spent a little while trying to get it running locally as an example however it seems slightly complicated and takes forever to get working.

If I should use Mesos, would I then have to develop a Framework for my project that takes all of my java tasks or are there existing solutions out there?

To test it on a small scale locally would you install it on your machine, set that to a master, create a VM, install it on that and make that a slave, somehow routing your slave to that master? The documentation and examples don't show how to exactly hook up a slave on the network to a master.

Thanks in advance, any help or suggestions would be appreciated.

Upvotes: 0

Views: 246

Answers (1)

hartem
hartem

Reputation: 411

You can definitely use Mesos for the task that you have described. You do not need to develop a framework from scratch, instead you can use a scheduler like Marathon in case you have long running tasks, or Chronos for one-off or recurring tasks.

For a real-life setup you definitely would want to have more than one machine, but you might as well just run everything (Mesos master, Mesos slave, and the frameworks) off of a single machine if you're only interested in experimenting. Examples section of Mesos Getting Started Guide demonstrates how to do that.

Upvotes: 2

Related Questions