CODEWITHSUNDEEP
.netdatatableperformance
qJake
qJake

Reputation: 17129

Efficiently extract a subset of rows from a large DataTable?

Given the following table, as an example:

+----+---------+-----------+
| ID | GroupID | OtherData |
+----+---------+-----------+
| 1  | 1       | w4ij6u    |
+----+---------+-----------+
| 2  | 2       | ai6465    |
+----+---------+-----------+
| 3  | 2       | ows64rg   |
+----+---------+-----------+
| 4  | 2       | wqoi46suj |
+----+---------+-----------+
| 5  | 3       | w9rthzv   |
+----+---------+-----------+
| 6  | 3       | 03ehsat   |
+----+---------+-----------+
| 7  | 4       | w469ia    |
+----+---------+-----------+
| 8  | 5       | nhwh57rt  |
+----+---------+-----------+
| 9  | 5       | mwitgjhx4 |
+----+---------+-----------+

How can I efficiently get a List<List<DataRow>> extracted from this table that is based upon the "GroupID" column?

Basically, I want the result to be:

MyList(0) = List: 1 DataRow, ID(s) 1
MyList(1) = List: 3 DataRows, ID(s) 2,3,4
MyList(2) = List: 2 DataRows, ID(s) 5,6
MyList(3) = List: 1 DataRow, ID(s) 7
MyList(4) = List: 2 DataRows, ID(s) 8,9

Here's the problem though: This DataTable contains hundreds of columns and tens of thousands of rows, so this operation must be as efficient as possible.

I have already tried the following methods:

I'm hoping someone else has a better, more efficient way of extracting this data.

Upvotes: 1

Views: 2722

Answers (2)

qJake
qJake

Reputation: 17129

Turns out that A) My returning data set was too large (because of a bug), and B) LINQ is probably the fastest way to do this, without writing some very long or hack-ish code. Thanks for your ideas, everyone, but I'll stick with LINQ for now.

Upvotes: 0

James Johnson
James Johnson

Reputation: 46047

Have you tried the DataTable.Select() method? Here's an example of how to use it:

DataTable table = GetSomeData();
DataRow[] results = table.Select("SomeInt > 0");

List<DataRow> resultList = results.ToList();

Using DataTable.Select should definitely be quicker than DefaultView.Filter, and as you can see the ability to put the results in a list is already built in.

Upvotes: 2

Related Questions