CODEWITHSUNDEEP
hadoopimportexporthbase
Radek Tomšej
Radek Tomšej

Reputation: 490

HBase Import command

We are currently migrating from CDH3u4 to CDH5. We made new cluster and copied all data. Everything went smooth thanks to Cloudera manager. But we have problem with migrating data from HBase 0.90.6 to HBase 0.96.1.1.

I tried to migrate data by using Export/Import feature of HBase (https://hbase.apache.org/book/ops_mgt.html#export. I have managed to export data and copy them to new server (discp). When I used command on destination cluster:

hbase -Dhbase.import.version=0.90 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3

Job was completed successfully, but no data was load (table ip is still blank):

14/06/25 09:04:58 INFO mapreduce.Job: Job job_1403615212297_0014 running in uber mode : false
14/06/25 09:04:58 INFO mapreduce.Job: map 0% reduce 0%
14/06/25 09:05:08 INFO mapreduce.Job: map 7% reduce 0%
14/06/25 09:05:11 INFO mapreduce.Job: map 43% reduce 0%
14/06/25 09:05:16 INFO mapreduce.Job: map 45% reduce 0%
14/06/25 09:05:18 INFO mapreduce.Job: map 50% reduce 0%
14/06/25 09:05:20 INFO mapreduce.Job: map 55% reduce 0%
14/06/25 09:05:21 INFO mapreduce.Job: map 57% reduce 0%
14/06/25 09:05:22 INFO mapreduce.Job: map 80% reduce 0%
14/06/25 09:05:23 INFO mapreduce.Job: map 86% reduce 0%
14/06/25 09:05:25 INFO mapreduce.Job: map 91% reduce 0%
14/06/25 09:05:26 INFO mapreduce.Job: map 98% reduce 0%
14/06/25 09:05:28 INFO mapreduce.Job: map 100% reduce 0%
14/06/25 09:05:28 INFO mapreduce.Job: Job job_1403615212297_0014 completed successfully
14/06/25 09:05:28 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=5172058
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=5452414893
HDFS: Number of bytes written=0
HDFS: Number of read operations=132
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Job Counters
Launched map tasks=44
Data-local map tasks=44
Total time spent by all maps in occupied slots (ms)=410004
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=410004
Total vcore-seconds taken by all map tasks=410004
Total megabyte-seconds taken by all map tasks=419844096
Map-Reduce Framework
Map input records=9964456
Map output records=0
Input split bytes=5720
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=7648
CPU time spent (ms)=117230
Physical memory (bytes) snapshot=17097363456
Virtual memory (bytes) snapshot=68115570688
Total committed heap usage (bytes)=26497384448
File Input Format Counters
Bytes Read=5452409173
File Output Format Counters
Bytes Written=0

When I look into log no error is here.

I would appreciate any idea, thank you so much!

Upvotes: 3

Views: 1864

Answers (2)

Radek Tomšej
Radek Tomšej

Reputation: 490

It seems the problem was in command:

hbase -Dhbase.import.version=0.90 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3

When I modified it like this, whole job went OK:

hbase -Dhbase.import.version=0.94 org.apache.hadoop.hbase.mapreduce.Import ip /user/rtomsej/ip3

Think that import.version=0.90 is not supported.

Upvotes: 3

user3132456
user3132456

Reputation: 21

I have the same problem, but your solution does not works for me.

I tried lots of time, found that once I disable the Table before running the Import task. There are error of "regionserver not online", but during the task is running, I enable the Table. The Import Task ended smoothly and the new data is loaded!!!

Upvotes: 0

Related Questions