CODEWITHSUNDEEP
javahbase
Deactivator2
Deactivator2

Reputation: 311

HBase LESS filter not working with Bytes

I'm trying to implement some functionality into an HBase query service, where I would be able to allow a user to query a guaranteed 'number' field by using the CompareOp.LESS operator. Here's a sample of the filter I use:

new SingleColumnValueFilter(
    Bytes.toBytes("TG-SMA"), 
    Bytes.toBytes("publishdate"),
    CompareOp.LESS, 
    Bytes.toBytes("20131201013434"));

Originally, I'd had the final parameter as

new BinaryComparator(Bytes.toBytes("20131201013434"))

but the behavior didn't change. What I'm getting in the logs as a result is this:

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Thu Dec 05 10:26:33 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:34 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:35 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:36 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:38 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:40 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:44 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:48 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:26:56 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException
Thu Dec 05 10:27:12 EST 2013, org.apache.hadoop.hbase.client.ScannerCallable@69822390, java.lang.NullPointerException

        at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:205)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:120)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:665)

I have another Filter that does work properly, for example:

new SingleColumnValueFilter(
    Bytes.toBytes("TG-SMA"), 
    Bytes.toBytes("title"),
    CompareOp.EQUAL, 
    new RegexStringComparator("This" + "\\.*"));

If I don't have that first filter, then I get correct results. I can also use this RegexStringComparator and CompareOp.EQUAL objects for the "publishdate" field from the first filter, and get correct results.

The way the code works, users can add multiple filters to query different columns/values. Currently, there are only these two filter types implemented, and only the second one works (the first one fails with the pasted error logs regardless of whether its the only filter present).

I'd also read about implementing my own custom Comparator/Comparable extension, but I do not believe I'm allowed to modify the HBase environment (this would be for a retail product), and implementing custom classes requires deploying to the HBase instance.

If anyone has any insight as to why I'd be seeing these errors, that would be fantastic and solve a lot of headaches. Thanks in advance.

Upvotes: 1

Views: 482

Answers (1)

Deactivator2
Deactivator2

Reputation: 311

Nevermind. Turned out that fields were being serialized before being submitted to HBase, and when the BinaryComparator would try to decode it, it would break on a null character. I'll leave this answer here as I never actually found an answer on the Internet, so hopefully this'll save someone a week's worth of diving down a rabbit hole.

The fix was to run the query values through the same serialization process, whatever that ends up being.

Upvotes: 1

Related Questions