How to load large amounts of data in a ListBox? ASP.NET MVC application

Question

I am working with an ASP.NET MVC application.

There is a requirement that a user be able to select an item from a ListBox that could contain over 30,000 entries.

Is there a dynamic way to populate the contents of this ListBox using an Ajax call - that would perform well?

Would I be better off just populating the ListBox control on the server and then having the user wait while the page renders with the 30,000 entries?

Would performance be better if I adopted some sort of jQuery solution?

Any suggestions on how to most efficiently deal with this scenario (without having the client change the requirement :-))?

Answer 1

I won't try to answer this as someone else already did:

This type of control is seen all too often in corporate applications: drop-down controls and lists that contain thousands of entries.

_{(source: mac.com)}

Programmers who mistakenly use it generally get a hint that it might be inappropriate when they find out that it takes an extraordinarily long time to load the form.

The following message, posted in a Visual Basic programmers forum on December 11, 1996, is typical:

I want to fill a list box with 2000 items ... This takes incredibly long ... over 20 minutes. Any ideas?

And another posted on December 16th, 1996, is somewhat less typical:

I'm looking for a list box control that can ... hold large numbers of entries (20,000+)

The excuse for such unwieldy controls is often a misguided interpretation of the almighty call to arms, "We must ensure data integrity." The programmers want to make sure that the user specifies a valid entry; in their view, the best way to do this is to force the user to select from a list. That would be fine if you had, say 20, 60, or maybe even 100 items in the list. Beyond that number, the fact that the user can only scroll a handful of items at a time causes the control to become unwieldy.

Imagine if you didn't have folders and directories on your hard drive. Whenever you needed to specify a file, you were presented with a drop-down control containing the name of every file on your hard drive, and asked to select the one you wanted to open. Few people, programmers included, would consider such a method as anything less than completely unacceptable.

All data can be organized in some meaningful way that will allow the user to more rapidly access the specific information he or she is interested in. Files are organized into folders or directories for example. Employees are often categorized by department, job title, or salary grade. Designing the interface to exploit the appropriate organization will allow the user to more rapidly locate the desired information, while at the same time, "ensuring data integrity."

Answer 2

I solved it by sending to user information that query returned more than 200 records as result and then I loaded just a half of them; on that way I suggest to user to add another filter in order to reduce number of records as result. Honestly speaking, I wish there is some better way to load hundred by hundred resords, as user scroll listbox down but no success yet.

Answer 3

Just use some good old fashioned validation; no need for a crazy listbox with every known option in the universe.

Answer 4

30,000 items isn't actually that much from a performance perspective. Give it a try, in testing I have been able to put up to 100,000 items.

From a users perspective though, I'd say 1,000 items at the absolute maximum is probably the limit from a usability perspective. I usually stop at 100 items. I'd suggest adding a textbox right above your listbox. Allow the user to search for terms. You can use jquery to populate the box, or use an update panel. If you limit the results to 100 items this will be really fast.

Answer 5

Sounds like a complete madness.

How would a user feel if he had to scroll through 30000 entries? Also, for users with not perfect eyes and somewhat undeveloped motor skills to precisely manipulate a mouse the experience will be pain.

Don't do it. Use other approaches:

• Paging of content, optionally with checkboxes to select items
• Sorting of content according to various criteria
• Categorize options for a user to first choose the right category and then select from a short list of options
• If the precise selection is not important, try using Ajax technique to show a list with suggestions to choose from
• Add a filter field for the user to define the pattern and let the rest just drop away

Another idea for your is to use an enhanced list supporting multiple columns and icons. Look at the Windows Explorer. It's much easier to select from a few columns with big icons than from a long list of minuscule text strings.

One more drawback of your approach: Having that many records at once in a select list will also expand the page size up to many megabytes. If you have the view state enabled, it will be quite slow and resource consuming.

Answer 6

The only answer to this question is: don't. Have you ever tried selecting something from a ListBox with 30,000 items? Has the client ever done this?

Update: I think the most efficient solution is to email a link to this page to your client, to give him a sense of the universal horror this question has evoked.

Answer 7

If you load everything up front, the performance is only going to be as slow as it takes to download the markup and render it initially.

If you load everything through AJAX calls (jQuery or otherwise), it's going to have to make the request, download the results, parse the results, and render them to the page...which is even slower (especially for something that size).

I would ask why the ListBox needs so many entries. No user is going to be able to effectively use that control. Maybe cut down the size of the list and implement paging to make things a little more usable?

Answer 8

This sounds like a bad idea - 30,000 entries? How would you feel if you had to select from a list this size?

It is better to use autocomplete for this type of use case.