CODEWITHSUNDEEP
c#multithreadingparallel-processing
deepak verma
deepak verma

Reputation: 21

run foreach loop via parallel to build result list

I am looping thro data from tblCustomer and checking if the address is valid or not. If not, then I am adding this to my return list. The problem, there are 37000 rows to validate. The validation is done via external library. It's taking about 1hr. I want to do this threaded so I can do it much faster. Can someone help me rewrite it? Also I read somewhere to wrap this in parallel class for or foreach. Few things that I am wondering -

  1. how many threads will it create? how can we control that?

  2. can we say how many records each thread will process?

  3. and I think the most imp question: this dll has static class that will validate the address. Is it going to give me any performance gain when I split it into threads..or will it take the same time?

 List<tblCustomer> customers = new List<tblCustomer>();
            int i = 0;
            foreach (var customer in DataContext.tblCustomers)
            {
                string addressToValidate = string.Format("{0}, {1}; {2} {3}", GetSafeString(customer.MailingCity), GetSafeString(customer.MailingState), GetSafeString(customer.MailingAddress), GetSafeString(customer.MailingAddress2));
                isTripValid = PCM.PCMSAddStop(tripId, addressToValidate.Trim()) == 1;
                if (!isTripValid)
                { 
                    customers.Add(customer);
                }
                i++;
                if (i == 1000)
                {
                    PCM.PCMSClearStops(tripId);
                    i = 0;
                }
            }
            PCM.PCMSCloseServer(serverID);
            PCM.PCMSDeleteTrip(tripId);

            return customers;

Upvotes: 2

Views: 355

Answers (1)

Mike Parkhill
Mike Parkhill

Reputation: 5551

  1. You can control the number of threads using the MaxDegreeOfParallelism property on the ParallelOptions class.

    ParallelOptions.MaxDegreeOfParallelism = 5; //Limit the concurrent threads to 5

  2. You can control the partitioning by using the ForEach() overload which takes a Partitioner<> object as a parameter.

    Here's a great article with an example on partitioning

  3. It probably will improve your performance still, but it's hard to know how much until you try it.

Note: if you're going to start using multiple threads make sure all of your shared objects/lists are threadsafe. For example, you'll need to either put a lock around your customers List or use a collection from the ConcurrentCollections namespace. Are the PCM methods threadsafe?

Upvotes: 1

Related Questions