CODEWITHSUNDEEP
cassandracqlscyllagocql
N G
N G

Reputation: 11

Gocql Allow Filtering returns empty records with page size

Im using gocql to interact with my scylla database and implementing pagination using page states. However while using ALLOW FILTERING it returns empty records if the filtered data is not within the defined page size.

This is a sample schema I'm using

CREATE TABLE table (
   partitonKeyOne        TEXT,
   partitonKeyTwo        TEXT,   
   clusteringKeyOne      TEXT,
   filterColumnOne       TEXT,
  
   PRIMARY KEY ((partitonKeyOne, partitonKeyTwo), clusteringKeyOne)
) WITH CLUSTERING ORDER BY (clusteringKeyOne DESC);

This is how I have implemented pagination using page state

getQuery := "SELECT * FROM table WHERE partitonKeyOne = 'abc' AND partitonKeyTwo = 'def' AND filterColumnOne = 'xyz' ALLOW FILTERING;"

queryResponse := db.GetSession().Query(
        getQuery,
    ).Consistency(gocql.One)

queryResponse.PageSize(limit)

queryResponse.PageState(pageState)

iterator := queryResponse.Iter()
nextPageState := hex.EncodeToString(iterator.PageState())

Now if the table has a total of 100 records and Im limiting each page size to 20 (i.e 5 pages in total) then if 60 records match the WHERE clause then I expect 20 records to be fetched from those 60 with an update pageState.

However, if there is not a single record matching the WHERE clause within the defined page size then it returns null

Upvotes: 1

Views: 185

Answers (1)

Nadav Har'El
Nadav Har'El

Reputation: 13791

This looks like a bug. Since exactly such a bug existed in the Python CQL driver several years ago (see https://github.com/scylladb/scylladb/issues/8203, fix in https://github.com/datastax/python-driver/commit/1d9077d3f4c937929acc14f45c7693e76dde39a9), and in the Cassandra Java driver (https://github.com/apache/cassandra-java-driver/pull/1544) - and I suspect the gocql driver has the same bug too.

I opened a new issue in Scylla's fork of the gocql driver (which I hope you're using): https://github.com/scylladb/gocql/issues/180, and I hope they'll fix it.

Scylla's test suite has a test, test_filtering_contiguous_nonmatching_partition_range, to verify that Scylla itself does not have this bug - that you can scan with filtering a long partition with all but the last row being filtered out, and the iteration doesn't stop on the first empty page - it continues until producing the match at the end. Because of this test I strongly suspect that the bug is in gocql, not in Scylla.

Upvotes: 0

Related Questions