CODEWITHSUNDEEP
hadoophbasehdfs
Shengjie
Shengjie

Reputation: 12806

Hbase memstore manual flush

According to Hbase design, Hbase uses memstore to store the writes and eventually when the memstore reaches the size limit, it flushes it to HDFS. This flushing exercise is happened automatically behind the theme.

In my case, I want to do a hdfs migration, migrate from one cluster to another, I need to make sure there is nothing left in-memory before I bring down hbase process in the source cluster. Is there anyway we can manually force the flush even tho the memstore hasn't reached the limit.

==question added==

further question: how do you know the flush is completed? via metrics?

Upvotes: 5

Views: 6995

Answers (2)

th30z
th30z

Reputation: 1942

From the shell you can just do flush 'tableName' to flush the memstore.

But if you want to do a backup of /hbase/table folder via hdfs, the way to do that is:

  • disable the table: (from the shell: disable 'tableName')
  • copy files: hadoop fs -cp /hbase/tableName /hbase-backup/tableName
  • enable the table: (from the shell: enable 'tableName')

...or you can use the CopyTable or Export tools (http://hbase.apache.org/book/ops.backup.html)

Upvotes: 13

linehrr
linehrr

Reputation: 1748

since you are going to migrate the hbase whole DB, you might want to do a batch disable:

disable_all '.*'

this will force hbase to flush out the memstore and write everything into HFiles. you will also notice that even after the disable, you will still see some WALs under /hbase/WALs, but don't worry, that's because hbase have a WAL ttl which keeps the WALs for a while even after flushing into HFiles.
to answer your question "how to verify flush is complete":
go to Hbase UI -> Regions -> Memory
you will see Memstore Size, make sure they are all "0"s.

Upvotes: 1

Related Questions