CODEWITHSUNDEEP
jvmcpu-usageapache-stormcodahale-metrics
paalaak
paalaak

Reputation: 183

Monitoring Storm JVM metrics

I have got a storm cluster running and I want to monitor its performance. I followed this blog and was able to measure the number of tuples received by a bolt using codahale metrics and display it in graphite.

My goal is to deploy a storm cluster on a lightweight computer such as beaglebone and for that I need to be able to monitor JVM parameters such as CPU, thread and memory usage of each Worker Process.

I really like codahale metrics and would like to continue using it in my application. Can anyone direct me as to how I can measure JVM parameters separately for each worker using codahale metrics?

I would really appreciate it if someone posted an example of how to get jvm metrics using codahale metrics.

Thanks,

Palak

Upvotes: 4

Views: 2996

Answers (3)

kartik
kartik

Reputation: 2135

Using VisualVM and JMX we can get the CPU usage,GC activity, class loading information, Heap size & Used Heap statistics, All the Threads information with statistics, CPU & Memory profiling, performance monitoring, Memory leaks of worker nodes. And also you can take heap dumps and thread dumps, profiler snapshots.

STEPS for setup

STEP 1: Staring VisualVM Java VisualVM is bundled with JDK version 6 update 7 or greater. Navigate to your JDK software's bin directory and double-click the Java VisualVM executable. Alternatively, navigate to your JDK software's bin directory and type the following command at the command (shell) prompt: jvisualvm.

STEP 2: Adding MBean plugin For JMX monitoring you need to add MBean plugin explicitly. 1, Choose Tools > Plugins from the main menu. 2, In the downloaded Plugins tab, Click Add Plugins 3, Select the Mbean plugin After successfully adding MBean plugin you can see MBean tab in VisualVM and you can monitor JMX.

STEP 3: Local Monitoring By default VisualVM will monitor all the applications running on the local JVM. No need to do any changes if your using Java 1.6 and above.

STEP 4: Remote Monitoring
To retrieve and display information on applications running on the remote host, the jstatd utility needs to be running on the remote host.

Steps to run jstatd
The jstatd tool is an RMI server application that monitors for the creation and termination of instrumented HotSpot Java virtual machines (JVMs) and provides an interface to allow remote monitoring tools to attach to JVMs.

1, create a file with "jstatd.all.policy" file name and copy the below content grant codebase "file:${java.home}/../lib/tools.jar" { permission java.security.AllPermission ;};

2, copy "jstatd.all.policy" file in java bin (Java\jdk1.7.0_10\bin) directory

3, Navigate to your JDK software's bin directory and type the following command at the command prompt: jstatd -J-Djava.security.policy=jstatd.all.policy.txt

4, to run jstatd admin privileges required, then only all the other users can connect it remote host. It’s one time activity. (Run with background process in CIT and SIT)

To add a remote host in VisualVM, right-click the Remote node in the Applications window, choose Add Remote Host and type the host name or IP address in the Add Remote Host dialog box. When Java VisualVM is connected to a remote host, a node for the remote host appears under the Remote node in the Applications window. You can expand the remote host node to view the applications running on the remote host.

Upvotes: 1

kartik
kartik

Reputation: 2135

Use jvisualvm.exe jdk/bin and you can monitor storm workers. Jvisualvm can also point to remote Storm topology.

Download and add mbean plugin into jvisualvm.

Upvotes: 0

paalaak
paalaak

Reputation: 183

I found an excellent tutorial here. Works like a charm.

Upvotes: 1

Related Questions