CODEWITHSUNDEEP
c#.netado.netparallel-processing
Jonesopolis
Jonesopolis

Reputation: 25370

Parallelism with Data from SqlDataReader

I have a loop that processes data, row by row from a SqlDataReader. It works dandy. Now I want to add parallelism to it. Ideally I want to read a row, toss it to a thread, read another row, etc. I'd like a configurable number of threads (say 10) so as one opens up (completes its task), another is allowed to start.

Is there a built in way to do this, or should I handle it myself? I looked at PLINQ and Parallel, but I have trouble getting my head around it i think.

Upvotes: 3

Views: 5541

Answers (1)

Mike Perrenoud
Mike Perrenoud

Reputation: 67898

Since you're using a DataReader, it's forward only and one row at a time in memory, so you're going to need to handle this on your own. You're going to need to marshal the row in memory into some data structure and pass that on to the thread and then issue Read() again to move next.

You'll have to determine how you're going to pause when you have too many threads open already, but that's a whole other discussion and if you end up having questions about that then post what you have and we'll gladly help!

Upvotes: 2

Related Questions