CODEWITHSUNDEEP
javascriptjqueryasp.net-mvcextjssproutcore
teleball
teleball

Reputation: 4662

Handling large grid datasets in JavaScript

What are some of the better solutions to handling large datasets (100K) on the client with JavaScript. In particular, if you have multi-column sort and search capabilities, how do you handle fetching (and pre-fetching) the data, client side model binding (for display), and caching the data.

I would imagine a good solution would be doing some thoughtful work in the background. For instance, initially, if the table was displaying N items, it might fetch 2N items, return the data for the user, and then go fetch the next 2N items in the background (even if the user hasn't requested this). As the user made search/sort changes, it would throw out (or maybe even cache the initial base case), and do similar functionality.

Can you share the best solutions you have seen?

Thanks

Upvotes: 3

Views: 9334

Answers (5)

Denis
Denis

Reputation: 2469

Check out this comprehensive list of data grids and spreadsheets.

For filtering/sorting/pagination purposes you may be interested in great Handsontable, or DataTables as a free alternative.

If you need simply display huge list without any additional features Clusterize.js should be sufficient.

Upvotes: 1

josh3736
josh3736

Reputation: 144932

SlickGrid does exactly what you're looking for. (Demo)

Using the AJAX data store, SlickGrid can handle millions of rows without flinching.

Upvotes: 3

Brian Moeskau
Brian Moeskau

Reputation: 20431

Since you tagged this with Ext JS, I'll point you to Ext.ux.LiveGrid if you haven't already seen it. The source is available, so you might have a look and see how they've addressed this issue. This is a popular and widely-used extension in the Ext JS world.

With that said, I personally think (virtually) loading that much data is useless as a user experience. Manually pulling a scrollbar around (jumping hundreds of records per pixel) is a far inferior experience to simply typing what you want. I'd much prefer some robust filtering/searching instead of presenting that much data to the user.

What if you went to Google and instead of a search box, it just loaded the entire internet into one long virtual list that you had to scroll through to find your site... :)

Upvotes: 2

DarrellNorton
DarrellNorton

Reputation: 4661

Use a jQuery table plugin like DataTables: http://datatables.net/

It supports server-side processing for sorting, filtering, and paging. And it includes pipelining support to prefetch the next x pages of records: http://www.datatables.net/examples/server_side/pipeline.html

Actually the DataTables plugin works 4 different ways: 1. With an HTML table, so you could send down a bunch of HTML and then have all the sorting, filtering, and paging work client-side. 2. With a JavaScript array, so you could send down a 2D array and let it create the table from there. 3. Ajax source - which is not really applicable to you. 4. Server-side, where you send data in JSON format to an empty table and let DataTables take it from there.

Upvotes: 3

Shanti
Shanti

Reputation: 1238

It depends on how the data will be used.

For a large dataset, where the browser's Find function was adequate, just returning a straight HTML table was effective. It takes a while to load, but the display is responsive on older, slower clients, and you never have to worry about it breaking.

When the client did the sorting and search, and you're not showing the entire table at once, I had the server send tab-delimited tables through XMLHTTPRequest, parsed them in the browser with list = String.split('\n'), and updated the display with repeated calls to $('node').innerHTML = 'blah'. The JS engine can store long strings pretty efficiently. That ran a lot faster on the client than showing, hiding, and rearranging DOM nodes. Creating and destroying new DOM nodes on the fly turned out to be really slow. Splitting each line into fields on-demand seems to work; I haven't experimented with that degree of freedom.

I've never tried the obvious pre-fetch & background trick, because these other methods worked well enough.

Upvotes: 1

Related Questions