CODEWITHSUNDEEP
apache-sparkhbase
Ravi Kiran G
Ravi Kiran G

Reputation: 467

Connection to remote hbase through scala spark

I am trying to connect to remote Hbase through scala and spark. Unable to succeeded. Can anyone suggest any methods related to this.

Thanks in advance.

Upvotes: 0

Views: 1462

Answers (2)

Water Guo
Water Guo

Reputation: 79

I came across similar problem last week. Eventually I made it using the HBase Spark connector. It is quite a bit setup/configuration. I've documented my steps in the link below

Setup Apache Zeppelin with Spark and HBase

Upvotes: 1

Augustine
Augustine

Reputation: 106

we have two methods to connect to HBASE from spark/scala

  1. HBASE Rest Api
  2. phoenix.apache -- https://phoenix.apache.org/

Hbase Rest API code

        val hbaseCluster  = new org.apache.hadoop.hbase.rest.client.Cluster()
                    hbaseCluster.add("localhost or UP", <port>)
                    val restClient = new Client(hbaseCluster)
                     val table =  new RemoteHTable(restClient, "STUDENT")
                     println("connected...")
                    var p = new Put(Bytes.toBytes("row1"))

              p.add(Bytes.toBytes("0"), Bytes.toBytes("NAME"),Bytes.toBytes("raju"))
              p.add(Bytes.toBytes("0"), Bytes.toBytes("COURSE"),Bytes.toBytes("SCALA"))
              p.add(Bytes.toBytes("0"), Bytes.toBytes("YEAR"),Bytes.toBytes("2017"))
              table.put(p)

              val scan = new Scan()
              val scanner : ResultScanner = table.getScanner(scan)
              println("got scanner...")
              val g = new Get(Bytes.toBytes("row1"))
              val result = table.get(g)

              val name = Bytes.toString(result.getValue(Bytes.toBytes("0"),Bytes.toBytes("NAME")))
              val course = Bytes.toString(result.getValue(Bytes.toBytes("0"),Bytes.toBytes("COURSE")))
              val year = Bytes.toString(result.getValue(Bytes.toBytes("0"),Bytes.toBytes("YEAR")))

              println("row1 " + "name: " + name + " course: " + course + " year:" + year);


              for (result  <- scanner) {
                      var userId = Bytes.toString(result.getValue("NAME".getBytes(), "ID".getBytes()))
                       println("userId " + userId)
               }
            }
            }

Apache Phoenix

Phoenix provides spark plugin and JDBC connection as well.

spark plugin - https://phoenix.apache.org/phoenix_spark.html

JDBC Connection (query server)- https://phoenix.apache.org/server.html

Upvotes: 2

Related Questions