CODEWITHSUNDEEP
javagwtsmartgwtlistgrid
Jon Taylor
Jon Taylor

Reputation: 7905

SmartGWT ListGrid large dataset problems

I have a SmartGWT ListGrid with 16 fields each containing Strings. The grid is attached to a data source provided via REST. I am facing issues though when the amount of data in the datasource is large.

When dealing with very small numbers there are no problems, even when dealing with 200,000 records there are still no problems. However when it comes to much larger datasets, for example one I tried earlier had 2.6 million records the grid only displays the first 850,000 (approximatley) records then refuses to page any further. Even more oddly, when my datasource has about 20 million records the grid only displays the first 20 then refuses to page at all.

Strangley I can see the data coming back in my RPC response using the isc.showConsole(); and can see that it is returning the correct data, in fact even when it returns 64 valid records it only displays the first 20. The totalRecords is properly returned as 20 million but it just will not page.

My grid is setup as follows

ListGrid grid = new ListGrid();
DataSource ds = Application.getMyDataSource();
grid.setCriteria(new Criteria("someid", 627263));
grid.setDataSource(ds);

grid.setAutoFetchData(true);

My Datasource is setup correctly as it works perfectly on smaller datasets, each field is created as a DataSourceTextField.

The server side accepts the incomming request, looks for the someid coming in, checks the _startRow, _endRow parameters finds those rows (I'm not using a database for this) and manually sets the startRow, endRow and totalRow fields.

When I inspect the data coming back the data is correct yet the grid refuses to display it correctly.

What am I missing? Is there a maximum size for list grids? From what I've seen they should handle millions of records, yet mine seems to fail completely.

EDIT

Upon further inspection it seems it is an error with the way SmartGWT handles the sizing of the scrollbar. I have no idea how to fix this at the moment.

When scrolling to the bottom of the table on what should be a 2.3 million row table, the last row displayed is 894,785. This is well short of the number of rows expected.

All data coming back from the data source is correct at this point. Interestingly clicking on the last row and scrolling with the down arrow on keyboard works fine, this to me re-enforces the point that it is a scroll bar issue.

Upvotes: 2

Views: 4493

Answers (1)

Sithsu
Sithsu

Reputation: 2219

In order for progressive loading to work, grid has to be tricked in a way to indicate there's more data.

Since we don't want to draw 1000 empty records (actual totalRows) when only 150 is fetched with data, we need to set totalRows = 160 or something to generate a close enough scroll thumb size, as well as allow scrolling below 150 records, to trigger progressive loading of next page.

Check:
Paging and total dataset length in ResultSet
http://forums.smartclient.com/showthread.php?t=516
http://forums.smartclient.com/showthread.php?t=2750

When you drag the scroll thumb too fast towards the bottom, grid might 'intelligently' skip loading intermediate rows.

ListGrid.showAllRecords might be what you need to check based on http://forums.smartclient.com/showthread.php?t=23638.

That last thread also indicates issues with browsers and large datasets.

Upvotes: 0

Related Questions