CODEWITHSUNDEEP
hadoophbase
Partha
Partha

Reputation: 570

HBase - hotspotting check

I am using HBase. And I am suspecting that rowkey has caused hotspoting. Before trying with salting of rowkey, I would like to check if hotspoting has already occurred. Is there any way in HBase to analyze data distribution in region servers to check if hotspoting has occurred?

Thanks, Partha

Upvotes: 0

Views: 1565

Answers (1)

Feni Say
Feni Say

Reputation: 11

You can use the HMaster Info Web UI to detect this. It should be http://master-address:16010 by default.

If it's not available, you can check if the UI is not disabled in the conf (hbase-site.xml) and be sure that hbase.master.info.port is not set to -1.

When you are on it, you have to click on the table that you want to check. You will be on this page

https://docs.prediction.io/images/cloudformation/hbase-32538c47.png

Then if you see that one region server has a lot more regions than the others, this is a good hint that one of your region server is probably hotspotted. It means that the regions in this part of the rowkey scope are splitted more often ! The request per second can also be an indicator but to my experience, it's not always really accurate.

But this is just good hints and the only simple good way that I know to be sure that a hotspot is occuring is to bench it. Because when it happens, the write performance are really, REALLY different. So, you should check the througput that you have with an hashed rowkey with the same data then compare. You'll see very quickly if there is an hotspot.

Upvotes: 0

Related Questions