CODEWITHSUNDEEP
c#.netforeachparallel.foreach
royu
royu

Reputation: 377

Why this difference between foreach vs Parallel.ForEach?

Can anyone explain to me in simple langauage why I get a file about 65 k when using foreach and more then 3 GB when using Parallel.ForEach?

The code for the foreach:

// start node xml document
var logItems = new XElement("log", new XAttribute("start", DateTime.Now.ToString("yyyy-MM-ddTHH:mm:ss")));
var products = new ProductLogic().SelectProducts();
var productGroupLogic = new ProductGroupLogic();
var productOptionLogic = new ProductOptionLogic();
// loop through all products
foreach (var product in products)
{
    // is in a specific group
    var id = Convert.ToInt32(product["ProductID"]);
    var isInGroup = productGroupLogic.GetProductGroups(new int[] { id }.ToList(), groupId).Count > 0;
    // get product stock per option
    var productSizes = productOptionLogic.GetProductStockByProductId(id).ToList();
    // any stock available
    var stock = productSizes.Sum(ps => ps.Stock);
    var hasStock = stock > 0;
    // get webpage for this product
    var productUrl = string.Format(url, id);
    var htmlPage = Html.Page.GetWebPage(productUrl);
    // check if there is anything to log
    var addToLog = false;
    XElement sizeElements = null;
    // if has no stock or in group
    if (!hasStock || isInGroupNew)
    {
        // page shows => not ok => LOG!
        if (!htmlPage.NotFound) addToLog = true;
    }
    // if page is ok
    if (htmlPage.IsOk)
    {
        sizeElements = GetSizeElements(htmlPage.Html, productSizes);
        addToLog = sizeElements != null;
    }
    if (addToLog) logItems.Add(CreateElement(productUrl, htmlPage, stock, isInGroup, sizeElements));
}
// save
var xDocument = new XDocument(new XDeclaration("1.0", "utf-8", "yes"), new XElement("log", logItems));
xDocument.Save(fileName);

Use of the parallel code is a minor change, just replaced the foreach with Parallel.ForEach:

// loop through all products
Parallel.ForEach(products, product =>
{
    ... code ...
};

The methods GetSizeElements and CreateElements are both static.

update1 I made the methods GetSizeElements and CreateElements threadsafe with a lock, also doesn't help.

update2 I get answer to solve the problem. That's nice and fine. But I would like to get some more insigths on why this codes creates a file that is so much bigger then the foreach solutions. I am trying get some more sense in how the code is working when using threads. That way I get more insight and can I learn to avoid the pitfalls.

Upvotes: 1

Views: 486

Answers (2)

mehmet mecek
mehmet mecek

Reputation: 2685

Try to define the following parameters inside the foreach loop.

var productGroupLogic = new ProductGroupLogic();
var productOptionLogic = new ProductOptionLogic();

I think the only two is used by all of your threads inside the parallel foreach loop and the result is multiplied unnecessaryly.

Upvotes: 1

Henk Holterman
Henk Holterman

Reputation: 273179

One thing stands out:

if (addToLog) 
  logItems.Add(CreateElement(productUrl, htmlPage, stock, isInGroup, sizeElements));

logItems is not tread-safe. That could be your core problem but there are lots of other possibilities.

You have the output files, look for the differences.

Upvotes: 2

Related Questions