CODEWITHSUNDEEP
javaspringhbase
tgkprog
tgkprog

Reputation: 4598

spring hbase sample with standalone hbase

On my local dev Ubuntu 14 box I cant get hadoop and hbase to run standalone.

BUt can get hbase to work directly with the file system (verfied by creating table and putting values in the hbase shell).

Now I want to read this table from spring. I started out with code at https://github.com/spring-projects/spring-hadoop-samples/tree/master/hbase

If I change the IP and port to the qa server it works. But if I want to connect to my local hbase not able to.

hbase-site

   <property>
      <name>hbase.rootdir</name>
      <!-- value>hdfs://localhost:50070/hbase</value --> <!-- 8030-->
    <value>file:///usr/local/hbase/HFiles3</value>
   </property>

   <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/usr/local/hbase/zookeeper3</value>
  </property>     

   <property>
    <name>hbase.master.info.port</name>
    <value>8095</value>
  </property>

   <property>
    <name>hbase.regionserver.info.port</name>
    <value>8096</value> <!-- changed as i dont know where it was connecting other wise none of the usual ports -->
  </property>

application-context.xml

    <context:property-placeholder location="hbase.properties"/>

    <context:component-scan base-package="org.springframework.samples.hadoop.hbase"/>

    <hdp:configuration id="hadoopConfiguration">
      fs.defaultFS=file:///usr/local/hbase/HFiles3
    </hdp:configuration>

<!--    configuration-ref="hadoopConfiguration" -->
    <hdp:hbase-configuration configuration-ref="hadoopConfiguration" zk-quorum="${hbase.zk.host}" zk-port="${hbase.zk.port}"/>

    <bean id="hbaseTemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate">
        <property name="configuration" ref="hbaseConfiguration"/>
    </bean>

hbase.properties

hbase.zk.host=localhost
hbase.zk.port=8095

Log (part): ...

12:20:21,763  INFO           org.apache.zookeeper.ZooKeeper: 100 - Client environment:user.home=/home/t
12:20:21,763  INFO           org.apache.zookeeper.ZooKeeper: 100 - Client environment:user.dir=/home/t/prog/r/github/spring-hadoop-samples/hbase
12:20:21,765  INFO           org.apache.zookeeper.ZooKeeper: 438 - Initiating client connection, connectString=localhost:8095 sessionTimeout=90000 watcher=hconnection-0x5a7fd55c, quorum=localhost:8095, baseZNode=/hbase
12:20:21,802  INFO oop.hbase.zookeeper.RecoverableZooKeeper: 120 - Process identifier=hconnection-0x5a7fd55c connecting to ZooKeeper ensemble=localhost:8095
12:20:21,805  INFO          org.apache.zookeeper.ClientCnxn: 975 - Opening socket connection to server t/127.0.0.1:8095. Will not attempt to authenticate using SASL (unknown error)
12:20:21,828  INFO          org.apache.zookeeper.ClientCnxn: 852 - Socket connection established to t/127.0.0.1:8095, initiating session
12:20:21,834  WARN          org.apache.zookeeper.ClientCnxn:1102 - Session 0x0 for server t/127.0.0.1:8095, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len1213486160 is out of range!
    at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
12:20:21,947  WARN oop.hbase.zookeeper.RecoverableZooKeeper: 253 - Possibly transient ZooKeeper, quorum=localhost:8095, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
12:20:21,947  INFO rg.apache.hadoop.hbase.util.RetryCounter: 155 - Sleeping 1000ms before retry #0...
12:20:23,325  INFO          org.apache.zookeeper.ClientCnxn: 975 - Opening socket connection to server t/127.0.0.1:8095. Will not attempt to authenticate using SASL (unknown error)
12:20:23,326  INFO          org.apache.zookeeper.ClientCnxn: 852 - Socket connection established to t/127.0.0.1:8095, initiating session
12:20:23,327  WARN          org.apache.zookeeper.ClientCnxn:1102 - Session 0x0 for server t/127.0.0.1:8095, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len1213486160 is out of range!

...

Can I run spring hbase without hadoop for local dev? Do I need to run a stand alone zookeeper and give that port as hbase.zk.port ? what is hbase.zk.port?

Upvotes: 0

Views: 1099

Answers (1)

tgkprog
tgkprog

Reputation: 4598

In standalone mode it connects to 2181 too. From the logs of hbase:

2015-08-01 12:30:36,489 INFO [main] server.ZooKeeperServer: Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /usr/local/hbase/zookeeper3/zookeeper_0/version-2 snapdir /usr/local/hbase/zookeeper3/zookeeper_0/version-2 2015-08-01 12:30:36,504 INFO [main] server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:2181 2015-08-01 12:30:36,638 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:60489

Sample worked - 'hadoop' local file URL and zk with default 2181 port

<hdp:configuration id="hadoopConfiguration">
      fs.defaultFS=file:///usr/local/hbase/HFiles3
    </hdp:configuration> 

hbase.zk.port=2181

Upvotes: 1

Related Questions