CODEWITHSUNDEEP
hadoophbaseapache-pighortonworks-data-platform
gauravbhide
gauravbhide

Reputation: 33

pig - read/write data from remote hbase server

I want to read/write data from hbase remote server from pig script. We are using Hortonworks HDP 2.5

Following is scenario.

We have two clusters, one for Hive and one for Hbase. We have access to edge node which is part of Hive cluster.

Our code (MapReduce jobs & Pig script) will be executed on Hive cluster.

As part of requirement, we have to also read/write data from Hbase tables.

We found one solution on net but it's not working. http://grokbase.com/t/cloudera/cdh-user/137at5cg72/pig-stores-into-remote-hbase

This solution suggests to register zookeeper quorum of hbase server inside our pig script. But the syntax is not working.

  set hbase.zookeeper.quorum '108.168.251.xxx-static.reverse.softlayer.com<http://108.168.251.xxx-static.reverse.softlayer.com:8020/hbase>'

...

STORE raw_rec INTO 'hbase://my_table<http://108.168.251.xxx-static.reverse.softlayer.com:8020/hbase>' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('score:val');

But this is not working. Getting error as only alphanumeric characters are allowed in namespace (< > sign not allowed)

Please let us know if there is any way to read/write data from remote hbase cluster inside pig using default org.apache.pig.backend.hadoop.hbase.hbasestorage

Thanks in Advance!!!

Upvotes: 1

Views: 429

Answers (1)

slimmilan
slimmilan

Reputation: 105

In the client machine where pig is installed, kindly set the below. Assuming 2181 is the zk cilent port.

export PIG_OPTS="$PIG_OPTS -Dhbase.zookeeper.property.clientPort=2181 -Dhbase.zookeeper.quorum=108.168.251.xxx-static.reverse.softlayer.com"

Then use table name directly without "hbase://" , as if my_table exists in the local cluster's hbase.

STORE raw_rec INTO 'my_table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('score:val');

Upvotes: 1

Related Questions