CODEWITHSUNDEEP
amazon-web-servicesapache-sparkhiveamazon-emrspark-thriftserver
James Wierzba
James Wierzba

Reputation: 17568

How to monitor hive thrift server health in spark hosted in AWS EMR cluster?

We use a JDBC to connect to the hive thrift server on our spark EMR cluster. We have had some transient issues of the thrift server crashing and requiring manual restart.

We are wondering if there is any [hopefully simple] way for us to monitor the health of the thrift server? At the very least, we would like to get alerted when it goes down. It would also be nice to have other server health metrics.

Upvotes: 0

Views: 1210

Answers (1)

Kaushal
Kaushal

Reputation: 3367

Hive thrift server is just a spark application and its also provide you Spark UI to monitor what all queries you are executing, memory consumption and much more information.

Default port for any spark UI is 4040. But if not.

Then do execute jps command on your master machine. it will list all java application with their corresponding process ids. And fter that execute netstat -ntpul | grep <process id>, which will list all ports related to that process.

Upvotes: 1

Related Questions