CODEWITHSUNDEEP
c#multithreadingsql-server-2008select
Jack
Jack

Reputation: 11

How to get select from SQL Server for multithreading?

I have a C# program that selects all of the rows of a table with about 100 million rows, and performs some analysis on each row. The analysis of each row is independent from that on every other row. I would like to make my app multithreaded in order to speed up computation (I am running on a dual-processor quad-core Intel Xeon). I would like, for two threads, that one thread select the first half (about 50 million) of the rows and that other thread select the second half of the rows. What is the most efficient way to do this? My rows all have primary ids. My program right now jsut runs select * from table.

Upvotes: 1

Views: 1393

Answers (4)

Scott Bruns
Scott Bruns

Reputation: 1981

Try running the analysis in the query itself. It might be a lot faster than returning the rows and running the analysis locally.

Upvotes: 1

Jeff Reddy
Jeff Reddy

Reputation: 5670

I think you need to determine where the bottleneck is. Is your select statement taking 10 seconds to execute, but your code analysis is taking milliseconds? Or is it the other way around?

I would put some metrics together for everything you have going on before attempting to jump head first in to multi-threading. I think you'll find your C# code is going to be very fast and that your SQL is going to slow you down. Look into optimizing your SQL and your database first.

Upvotes: 1

michael
michael

Reputation: 11

Why not just use this?

select * from table where id % 2 = 0
select * from table where id % 2 = 1

Upvotes: 1

skaz
skaz

Reputation: 22580

Are you using LINQ and .NET 4.0? If so, you can leverage Parallel Linq to do what you want easily.

Upvotes: 2

Related Questions