CODEWITHSUNDEEP
javaperformancehadoophbasebatch-processing
TigerShark
TigerShark

Reputation: 13

HBase read : To improve on performance , how to do batch processing of get request using hbase java REST api

I am new to HBase REST API and was trying to find a way, that I could get a set of results for a set of Id's I would be doing a with a get command. As batch processing would help improve the performance of my code, instead of making a get request for each id for a table.

Some example Hbase java rest api code would be helpful.

thanks, in advance.

Upvotes: 1

Views: 2527

Answers (3)

Ben Watson
Ben Watson

Reputation: 5541

Result[] results = table.get(List<Get> gets)

does what you're looking for. You should see great performance improvements.

If you just want to know whether keys exist:

boolean[] exists = exists(List<Get> gets);

which can be even faster than get as it only returns true or false.

Upvotes: 2

vvg
vvg

Reputation: 6385

You can perform set of Get requests individually for each ID.

curl -vi -X GET \
         -H "Accept: text/xml" \
         "http://example.com:20550/users/row1"

Or create filter and perform request with specified filter:

curl -vi -X PUT \
         -H "Accept: text/xml" \
         -H "Content-Type:text/xml" \
         -d @filter.txt \
         "http://example.com:20550/users/scanner/"

<Scanner batch="100">
  <filter>
    {
      "type": "PrefixFilter",
      "value": "row_id_starts_with"
    }
  </filter>
</Scanner>

More about REST API in cloudera docs: https://www.cloudera.com/documentation/enterprise/5-9-x/topics/admin_hbase_rest_api.html

Upvotes: 0

TheCodingFrog
TheCodingFrog

Reputation: 3514

You should able to achieve it using scanner (HBase search API). Below e.g.

Scan scan = new Scan();
scan.setTimeRange( lowerBound, upperBound );

 Call it for each Column
scan.addColumn //

scan.setCaching( 1000 ) // how many rows for caching that will be passed to scanners.
ResultScanner scanner = table.getScanner( scan );
Iterator< Result > iterator = scanner.iterator();

There are plenty of articles available to get more detail e.g. https://www.cloudera.com/documentation/enterprise/5-4-x/topics/admin_hbase_scanning.html

Upvotes: 0

Related Questions