CODEWITHSUNDEEP
cassandradatastax-java-drivercqlshspring-data-cassandra
Cory
Cory

Reputation: 929

How to implement pagination for cassandra by using keys?

I'm trying to implement some kind of pagination feature for my app that using cassandra in the backend.

CREATE TABLE sample (
    some_pk int,
    some_id int,
    name1 txt,
    name2 text,
    value text,
    PRIMARY KEY (some_pk, some_id, name1, name2)
)
WITH CLUSTERING ORDER BY(some_id DESC)

I want to query 100 records, then store the last records keys in memory to use them later.

+---------+---------+-------+-------+-------+
| sample_pk| some_id | name1 | name2 | value |
+---------+---------+-------+-------+-------+
| 1       | 125     | x     | ''    | ''    |
+---------+---------+-------+-------+-------+
| 1       | 124     | a     | ''    | ''    |
+---------+---------+-------+-------+-------+
| 1       | 124     | b     | ''    | ''    |
+---------+---------+-------+-------+-------+
| 1       | 123     | y     | ''    | ''    |
+---------+---------+-------+-------+-------+

(for simplicity, i left some columns empty. partition key(sample_pk) is not important)

let's assume my page size is 2.

select * from sample where sample_pk=1 limit 2;

returns first 2 rows. now i store the last record in my query result and run query again to get next 2 rows;

this is the query that does not work because of restriction of a single non-EQ relation

select * from where sample_pk=1 and some_id <= 124 and name1>='a' and name2>='' limit 2;

and this one returns wrong results because some_id is in descending order and name columns are in ascending order.

select * from where sample_pk=1 and (some_id, name1, name2) <= (124, 'a', '') limit 2;

So I'm stuck. How can I implement pagination?

Upvotes: 0

Views: 4683

Answers (2)

Mohamed Ibrahim Elsayed
Mohamed Ibrahim Elsayed

Reputation: 2964

You don't have to store any keys in memory, also you don't need to use limit in your cqlsh query. Just use the capabilities of datastax driver in your application code for doing pagination like the following code:

public Response getFromCassandra(Integer itemsPerPage, String pageIndex) {
    Response response = new Response();
    String query = "select * from sample where sample_pk=1";
    Statement statement = new SimpleStatement(query).setFetchSize(itemsPerPage); // set the number of items we want per page (fetch size)
    // imagine page '0' indicates the first page, so if pageIndex = '0' then there is no paging state
    if (!pageIndex.equals("0")) {
        statement.setPagingState(PagingState.fromString(pageIndex));
    }
    ResultSet rows = session.execute(statement); // execute the query
    Integer numberOfRows = rows.getAvailableWithoutFetching(); // this should get only number of rows = fetchSize (itemsPerPage)
    Iterator<Row> iterator = rows.iterator();
    while (numberOfRows-- != 0) {
        response.getRows.add(iterator.next());
    }
    PagingState pagingState = rows.getExecutionInfo().getPagingState();
    if(pagingState != null) { // there is still remaining pages
        response.setNextPageIndex(pagingState.toString());
    }
    return response;
}

note that if you make the while loop like the following:

while(iterator.hasNext()) {
    response.getRows.add(iterator.next());
}

it will first fetch number of rows as equal as the fetch size we set, then as long as the query still matches some rows in Cassandra it will go fetch again from cassandra till it fetches all rows matching the query from cassandra which may not be intended if you want to implement a pagination feature

source: https://docs.datastax.com/en/developer/java-driver/3.2/manual/paging/

Upvotes: 1

Raj Parekh
Raj Parekh

Reputation: 192

You can run your second query like,

select * from sample where some_pk =1 and some_id <= 124 limit x;

Now after fetching the records ignore the record(s) which you have already read (this can be done because you are storing the last record from the previous select query).

And after ignoring those records if you are end up with empty list of rows/records that means you have iterated over all the records else continue doing this for your pagination task.

Upvotes: 2

Related Questions