CODEWITHSUNDEEP
hbase
user1326784
user1326784

Reputation: 657

Load data into hbase

How to load data which is in any format(like:json,xml,etc) into a hbase table ?Is there any specific output format which can be set in job config(java),so that any form of data can be loaded into hbase or is there any apps which can internally load any form of data into hbase?

Upvotes: 1

Views: 1335

Answers (2)

Arnon Rotem-Gal-Oz
Arnon Rotem-Gal-Oz

Reputation: 25909

When you want to store the data in HBase you need to make some additional choices it isn't just a file. for instance you need to decide what the key is, what column families you'd have, their characteristics (e.g. compressed, with TTL etc.) and if you store all the input in a single column or you parse it and store fragments in different columns.

This means that you'd have to do some processing before you store the data and it isn't just a job config.

That said, when you want to create a job that would write to HBase you can tell it which table/tables are involved via the TableMapReduceUtil as in:

    Job job = new Job(conf, "My Job");
    job.setJarByClass(Mymapred.class);

    Scan scan = new Scan();
    // set the scan parameters ..

    TableMapReduceUtil.initTableMapperJob(
            INPUT_TABLE_NAME,
            scan,
            MyMapper.class,Text.class,Result.class,
            job);

    TableMapReduceUtil.initTableReducerJob(
            OUTPUT_TABLE_NAME,
            MyReducer.class,
            job);

Upvotes: 0

Jayan Kuttagupthan
Jayan Kuttagupthan

Reputation: 550

The data you are inserting to hbase tables should be in bytes.So, even if it is in XML or JSON, you should convert the same into bytes. The reverse logic should be applied while retrieving data from hbase tables. A Utility class containing the conversion logic will do the job.

Upvotes: 2

Related Questions