CODEWITHSUNDEEP
hadoophive
이대희
이대희

Reputation: 65

How to insert data from file into HBase table?

I made example.nt which looks like below.

    1 "aaaaa1" "bbbbb1" "ccccc1"
    2 "aaaaa2" "bbbbb2" "ccccc2"
    3 "aaaaa3" "bbbbb3" "ccccc3"
    .......

I want insert this data into HBase table which consist of

    (key int,subject string,predicate string,object string)
    (:key,cf1:val1,cf1:val2,cf1:val3)

I want perform this inserting on the hbase shell. How can I do this?

Upvotes: 1

Views: 4558

Answers (1)

maxteneff
maxteneff

Reputation: 1531

HBase shell is not designed for these purposes, it allows insert data to HBase only line by line with put commands.

Instead of this you can use importtsv tool which allows you import text data directly to HBase.

Assuming you have already created HBase table so_table with one column family cf1 and your example.nt file is in the /tmp/example/ HDFS directory. So it's possible to use it by the following way:

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=HBASE_ROW_KEY,cf1:val1,cf1:val2,cf1:val3 so_table /tmp/example/

May be you will need add option to change column separator:

-Dimporttsv.separator=';'

Furthermore you should understand that this way data inserts to HBase directly via many put command. There is another way to use importtsv tool which is well suitable for bulk loading big amounts of input data. You can generate StoreFiles and then load it entirely to HBase with completebulkload tool:

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.bulk.output=/tmp/example_output -Dimporttsv.columns=HBASE_ROW_KEY,cf1:val1,cf1:val2,cf1:val3 so_table /tmp/example/
hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles /tmp/example_output so_table

You can read official documentation of this tool: https://hbase.apache.org/book.html#_importtsv

Upvotes: 3

Related Questions