CODEWITHSUNDEEP
c#foreachparallel-processing
Jack
Jack

Reputation: 387

Local variables in parallel foreach loops c#

I had two nested foreach loops that need to run through a significant amount of data and calculation. Regular foreach loops took far too long (several hours).

So, I looked up ways to speed it up and found Parallel.ForEach. This is my first time dealing with parallelisation but the examples seem easy enough.

Below is my code currently, the problem with this is local variables (I think, at least). errors are added for nodes that work fine outside of the parallel loops.

Parallel.ForEach(allNodes, (startNode) =>
{
    Parallel.ForEach(allNodes, (endNode) =>
    {
        if (startNode != endNode)
        {
            List<Geo_Model_Struct> route = pathfinder.getRouteOptimised(startNode, endNode);

            if (route.Count <= 0)
            {
                //failed to find route
                errors.Add(string.Format("Cound not find a route from {0} to {1}", startNode, endNode));
            }
            else
            {
                List<Geo_Model_Struct> accessibleRoute = accessiblePathfinder.getRouteOptimised(startNode, endNode);
                if (accessibleRoute.Count <= 0)
                {
                    //failed to find route
                    errors.Add(string.Format("Cound not find an accessible route from {0} to {1}", startNode, endNode));
                }
            }            
        }
        endCount++;
        System.Diagnostics.Debug.WriteLine("I: {0}/{1}\tJ: {2}/{3}", startCount, allNodes.Count - 1, endCount, allNodes.Count - 1);
    }
  );
    startCount++;
});

I'm guessing it's something to do with the route local variable being altered when it shouldn't as nearly all checked routes fail. But I don't know how to reliably debug this kind of thing so any help is appreciated.

Edit:

I am testing all possible routes to make sure they all work. route.Count should be > 0 for most tests. When using traditional foreach loops this is the case (e.g. 15 out of 500 times route.Count <= 0 is true)

When using Parallel.ForEach route.Count is 0 most of the time (e.g. somewhere in the region of 494 out of 500 times) so very few actually pass the test and when looking at the errors produced most that fail in parallel, pass using the traditional foreach

Solved

I found a way to remove the need to get data from the database within the getRouteOptimised method. This fixed the issue. Still not sure exactly what it is about the db connection that caused the problem, but it works now.

Upvotes: 1

Views: 988

Answers (1)

user3956566
user3956566

Reputation:

Without seeing the rest of your code, I suspect the issue is with the pathfinder and accessiblepathfinder objects. They may not be thread safe. A possible way to circumvent this is to create those variables locally within the inner foreach loop.

if (startNode != endNode)
{
    // Create and Initialise pathfinder here
    MyPathFinderObject pathfinder = new MyPathFinderObject(<parameters>);
    List<Geo_Model_Struct> route = pathfinder.getRouteOptimised(startNode, endNode);

    if (route.Count <= 0)
        .../...
    else
    {                        
        // Create and Initialise accessiblePathfinder here
        MyAccessiblePathFinderObject accessiblePathfinder = new MyAccessiblePathFinderObject(<parameters>);
        List<Geo_Model_Struct> accessibleRoute = accessiblePathfinder.getRouteOptimised(startNode, endNode);
        .../...
    }            
}

However there is no guarantee that this will work.

From the docs:

You must be extremely cautious when getting data from properties and methods. Large object models are known for sharing mutable state in unbelievably devious ways.

Upvotes: 1

Related Questions