CODEWITHSUNDEEP
jrubyhbase
Jasper
Jasper

Reputation: 2471

HBase Shell iterating over & truncating tables

Trying to truncate tables like this 

list.each{|name| truncate(name) if name.end_with?('abc123')}

won't work. How would one approach this?

Upvotes: 3

Views: 2680

Answers (5)

Prasad Khode
Prasad Khode

Reputation: 6739

if you are in hbase shell then you can issue the following command to truncate all the tables data

list.each {|tableName| truncate tableName}

Upvotes: 1

slm
slm

Reputation: 16416

In Bash

I typically just use this Bash one-liner that I've developed to iterate over all the HBase tables and then truncate them one by one.

$ for i in $(hbase shell <<<list |& sed -e '/row(s)/,$d;1,/TABLE/d;/SLF4J:/d'); do \
    hbase shell <<<"truncate '$i'"; done

The for loop goes through all the output generated by this command:

hbase shell <<<list |& sed -e '/row(s)/,$d;1,/TABLE/d;/SLF4J:/d'

This takes the output from the hbase shell list command and then parses it down so that it's just the table names. The for loop then simply calls hbase shell <<<"truncate '..tablename..'" multiple times.

In Ruby

In HBase's shell you have full access to Ruby so you could do this instead.

$ echo 'list.each {|t| truncate t}; quit;' | hbase shell

Upvotes: 1

Lorand Bendig
Lorand Bendig

Reputation: 10650

In HBase shell you can also do:

java_import org.apache.hadoop.hbase.client.HBaseAdmin
java_import org.apache.hadoop.hbase.HBaseConfiguration

admin = HBaseAdmin.new(HBaseConfiguration.create)
admin.listTables.each {|i| t=i.getNameAsString; truncate(t) if t.end_with?('abc123')}

Upvotes: 4

Jasper
Jasper

Reputation: 2471

To complement Suman's answer from above, here is my own ruby rake task using thrift.

# coding: utf-8
namespace :truncate do
  desc 'batch truncate tables'
  task :tables => :environment do

    require 'thrift'

    socket = ::Thrift::Socket.new(Constants.hbase_url, Constants.hbase_port, 5)
    transport = ::Thrift::BufferedTransport.new(socket)
    transport.open

    protocol = ::Thrift::BinaryProtocol.new(transport)

    hbase_conn = Apache::Hadoop::Hbase::Thrift::Hbase::Client.new(protocol)

    # define some columns
    col1 = Apache::Hadoop::Hbase::Thrift::ColumnDescriptor.new(:name => 'cf1', :maxVersions => 1, :inMemory => true)
    col2 = Apache::Hadoop::Hbase::Thrift::ColumnDescriptor.new(:name => 'cf2', :maxVersions => 1, :inMemory => true)

    hbase_conn.getTableNames.each do |table_name|
      if table_name.end_with?('abc123')
        hbase_conn.disableTable(table_name)
        hbase_conn.deleteTable(table_name)
        hbase_conn.createTable(table_name,[col1,col2])
        puts "truncated #{table_name}"
      end
    end

  end
end

Upvotes: 1

Suman
Suman

Reputation: 9571

You can use Python's Happybase and the script below to do this (you need to enable a Thrift connection to HBase as well):

import happybase

# String that the table name ends with
string = "abc123"

# Connect to HBase
c = happybase.Connection()

# For each table
for t in c.tables():
  # If the table ends with this string
  if t.endswith(string):
    # Disable and delete the table
    c.disable_table(t)
    c.delete_table(t)
    # Recreate it
    # Make sure to edit the below line with your own column-family structure
    c.create_table(t, {'cf':{}})
    # Print the name of the table
    print (t + " truncated")

Upvotes: 2

Related Questions