Edge Node hortonworks usage

Question

I have a 6 Nodes (2 masters + 4 slaves)production cluster with HA configured .

The actual topology is :

• Master 1 :

  • Active HBase Master
  • Hive Metastore
  • HiveServer2
  • HST Server
  • Knox Gateway
  • Active NameNode
  • Oozie Server
  • Active ResourceManager
  • WebHCat Server
  • ZooKeeper Server
  • HST Agent
  • JournalNode
  • Metrics Monitor

• Master 2 :

  • App Timeline Server
  • Standby HBase Master​​
  • History Server
  • Infra Solr Instance
  • Metrics Collector
  • Grafana
  • Standby NameNode
  • Standby ResourceManager
  • Spark2 History Server
  • Zeppelin Notebook
  • ZooKeeper Server
  • HST Agent
  • JournalNode
  • Metrics Monitor
  • Clients

• SLAVE 1/2/3 :

  • DataNode
  • RegionServer
  • HST Agent
  • NodeManager

  • MetricsMonitoring

  • One of the slave nodes Contains : JournalNode + Zookeeper Server

Now We are planning to add some Edge Node .

Our plan is :

• SQL Edge Node :
  • HCatalog
  • HiveServer2
  • WebHCat
• Admin Edge Node
  • Ambari Server
  • Ranger
• Lineage Edge Node
  • Job History Server
  • Spark2 History Server
  • App Timeline Server
  • Slider Registry Server
• End User Access Edge Node
  • Hue
• Knox Edge Node
  • Knox Gateway
• Scheduling Edge Node
  • Oozie Server
  • Falcon

What do you think ?

What's the best practice ?

What's the components to move from Master/Slave to Edge nodes ?

Thanks

Answer 1

Edge nodes are meant to be Clients only. No masters/slaves. Very minimal resources other than disk space maybe for being to SCP files before using hdfs dfs -put

The Knox Gateway itself is somewhat self-described as a secure edge-node, proxy into the cluster. Depending on if you are actually using it.

If you aren't using HBase & Zeppelin, then, you could probably remove those from the cluster. If you have the available resources, HBase should sit on its own dedicated servers

Same for Zookeeper - those should ideally be separated for optimal throughput.