CODEWITHSUNDEEP
apache-zookeeperhigh-availabilitymesos
smeeb
smeeb

Reputation: 29497

VM architecture for running HA Mesos Cluster

I'm reading the Mesos Architecture docs which, ironically, don't actually specify which components are supposed to run on which VMs/physicals.

It looks like, to run Mesos in HA, you need several categories of components:

  1. Mesos Masters
  2. ZooKeeper instances (quorum)
  3. Hadoop clusters (job nodes? name nodes?)

But there's never any mention of how many you need of each type.

So I ask: How many VMs/physicals do you need to run Mesos with HA, and what components should be deployed to each?

Upvotes: 0

Views: 143

Answers (1)

Tobi
Tobi

Reputation: 31479

Did you have a look at the HA docs? To run Mesos in HA, you'll need the Mesos Masters and ZooKeeper. Any Hadoop-related configurations are out of scope for Mesos HA itself.

To have a HA setup, you'll need a uneven number of nodes for the Masters and ZooKeeper (because of quorum mechanism). In our case, we're running 3 Master and 3 ZooKeeper nodes on 3 machines (one Master and one ZooKeeper instance per machine), and a number of Mesos Slaves/Agents on different machines.

Theoretically, the Slaves/Agents can run on the same machines as the Masters/ZooKeepers as well. I guess this is a matter of preferences and availability of machines, and your SLA needs.

If you want to run a large-scale production setup, it will probably make a lot of sense to even separate the Master and ZooKeeper instances.

Further references:

Upvotes: 1

Related Questions