CODEWITHSUNDEEP
cassandrajmxprometheus
Firdousi Farozan
Firdousi Farozan

Reputation: 882

Cassandra Read or Write latency JMX metric

We are attempting to use Prometheus to manage Cassandra cluster (v2.1.x) in my organization. I have a question related to Read or Write latency metrics.

For a node level read or write latency, the metric that I am using is - org.apache.cassandra.metrics<type=(ClientRequest), scope=(Read|Write), name=(Latency)><>(OneMinuteRate)

For a table level, I am using - org.apache.cassandra.metrics<type=(Keyspace), keyspace=(\S*), name=(ReadLatency|WriteLatency)><>(OneMinuteRate)

I tested the metrics by running cassandra-stress utility on a sample table. At the table level, I am seeing that the writes are roughly 5000 per sec. This is as expected.

But the node level writes are around 100 per sec. My requirement is to graph node level writes per second and table level writes per sec.

Upvotes: 2

Views: 1983

Answers (1)

Simon Fontana Oscarsson
Simon Fontana Oscarsson

Reputation: 2134

Yes, ClientRequest Latency metric is on node level. The reason you only have 100 writes per second there means that this node is getting very few client requests.

The table level is for all nodes in the cluster, meaning all your nodes together are doing 5000 writes to this table.

You find out if you have any hotspots (meaning these nodes handle more traffic than others). Perhaps you are using a limited amount of partition keys which would target a limited amount of nodes. This is most likely your issue (poor data model). You can also check what Load Balancing Policy your driver is using and change that.

Upvotes: 1

Related Questions