CODEWITHSUNDEEP
apachehadoopnosqlhbase
vishnu
vishnu

Reputation: 51

Master startup cannot progress, in holding-pattern until region onlined

I've setup a Hbase cluster with two nodes and I've noticed the warning "AssignmentManager: STUCK Region-In-Transition" which is not allowing the master to startup.

Why it is happeneing and how to fix it?

Under Hbase UI I can see the below message.

b94eb458bf643b46deaf6b00998d1f95 hbase:namespace,,1542792846910.b94eb458bf643b46deaf6b00998d1f95. state=OPENING, ts=Wed Nov 21 09:39:46 UTC 2018 (PT18M9.696S ago), server=knewshoe.corp.com,16020,1542792833282

Logs:

2018-11-21 09:40:45,900 INFO  [ReadOnlyZKClient-observepreserve.corp.com:2181@0x4068418f] zookeeper.ZooKeeper: Session: 0x167359e5ad60006 closed
2018-11-21 09:40:45,900 INFO  [ReadOnlyZKClient-observepreserve.corp.com:2181@0x4068418f-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x167359e5ad60006
2018-11-21 09:40:49,266 WARN  [master/observepreserve:16000:becomeActiveMaster] master.HMaster: hbase:namespace,,1542792846910.b94eb458bf643b46deaf6b00998d1f95. is NOT online; state={b94eb458bf643b46deaf6b00998d1f95 state=OPENING, ts=1542793186164, server=knewshoe.corp.com,16020,1542792833282}; ServerCrashProcedures=false. Master startup cannot progress, in holding-pattern until region onlined.
2018-11-21 09:41:46,095 WARN  [ProcExecTimeout] assignment.AssignmentManager: STUCK Region-In-Transition rit=OPENING, location=knewshoe.corp.com,16020,1542792833282, table=hbase:namespace, region=b94eb458bf643b46deaf6b00998d1f95
2018-11-21 09:41:53,267 WARN  [master/observepreserve:16000:becomeActiveMaster] master.HMaster: hbase:namespace,,1542792846910.b94eb458bf643b46deaf6b00998d1f95. is NOT online; state={b94eb458bf643b46deaf6b00998d1f95 state=OPENING, ts=1542793186164, server=knewshoe.corp.com,16020,1542792833282}; ServerCrashProcedures=false. Master startup cannot progress, in holding-pattern until region onlined.

Upvotes: 3

Views: 3530

Answers (2)

user7166292
user7166292

Reputation: 3

I had the problem like this: WARN [...host:16000:becomeActiveMaster] master.HMaster (HMaster.java:isRegionOnline(1350)) - hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPEN, ts=1720351000617, server=host.localdomain,16201,1719929681295}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined.

Solved by:

  1. zookeeper-shell.sh localhost

deleteall /hbase

  1. hadoop fs -rm -r /hbase

Upvotes: 0

Matiji66
Matiji66

Reputation: 737

Yes,reinstall hbase cause this problem! This is because old metadata was not removed,you need try to delete hbase metadata from zk and restart hbase,all will be ok,good luck.

Upvotes: 0

Related Questions