CODEWITHSUNDEEP
javaspring-batch
Ray
Ray

Reputation: 547

Spring Batch JdbcPagingItemReader paging not work

I use spring batch to do a data migration job. there are a lot of data so that I decided to use the JdbcPagingItemReader to read the data by page. Below is how I define the reader:

private JdbcPagingItemReader<Map<String, Object>> buildItemReader(final DataSource dataSource, String tableName,
        String tenant){

    String tenantName = tenantHelper.determineTenant(tableName);

    Map<String, Object> sqlParameterValues = new HashMap<>();
    sqlParameterValues.put("tableName", tableName);
    sqlParameterValues.put("tenantName", tenantName);
    sqlParameterValues.put("tenant", tenant);

    JdbcPagingItemReader<Map<String, Object>> itemReader = new JdbcPagingItemReader<>();
    itemReader.setDataSource(dataSource);
    itemReader.setPageSize(2);
    itemReader.setFetchSize(2);
    itemReader.setQueryProvider(generateSqlPagingQueryProvider(tableName,tenantName,tenant));
    //itemReader.setParameterValues(sqlParameterValues);
    itemReader.setRowMapper(new ColumnMapRowMapper());
    try {
        itemReader.afterPropertiesSet();
    } catch (Exception e) {
        e.printStackTrace();
    }

    return itemReader;
}

private PostgresPagingQueryProvider generateSqlPagingQueryProvider(String tableName, String tenantName,
        String tenant) {
    PostgresPagingQueryProvider provider = new PostgresPagingQueryProvider();
    Map<String, Order> sortKeys = new LinkedHashMap<>();
    String sortKey = getSortKeyBytable(tableName);
    sortKeys.put(sortKey, Order.ASCENDING);

    provider.setSelectClause("select *");
    provider.setFromClause("from " + tableName);
    provider.setWhereClause("where " + tenantName + " ='" + tenant + "'");
    provider.setSortKeys(sortKeys);
    return provider;
}

the sortkey I specified is the primary key in the table, it is a string. But the paging is not working as expected. But it throws no error it just read all the data.

In the spring batch document, it provides an example which use the int type id as the sort key, I am wondering if the spring batch paging read only support the int type sort key word? And can not support the string sort key?

Is this a limitation of spring batch?

Upvotes: 2

Views: 3374

Answers (2)

Ray
Ray

Reputation: 547

The above code in the question did work and solve my memory issue, the reason it did not work is because I still calling the Cursor Reader in my spring batch step defination. So the paging reader can solve the memory issue

Upvotes: 0

Anirudh Simha
Anirudh Simha

Reputation: 360

Its not a limitation of spring batch. This is how a database works. It sorts string based on its ascii value.

Upvotes: 2

Related Questions