Request terminates prematurely in .NET MVC 4, probably due to the multi-threading work

Question

In the action method, I'm trying to generate separate report for each contract, zip all the reports and send the zip file back to client.

To improve the performance, I use ThreadPool.QueueUserWorkItem method to multi-threading the tasks. To wait for all the work done before sending back the zip file, WaitHandle.WaitAll method is used here.

It works well with small datasets. For example, one 9-contract dataset took 1.5 mins for the whole process. But for a large dataset that has 86 contracts, the response comes back immediately after I send the request. It's a invalid zip file. I can tell the threads are still running to generate reports. But WaitHandle.WaitAll and anything after are not executed at all, as far as I see.

Any idea why?

public void GenerateReport(ReportParams parameters)
{
        System.Web.HttpContext.Current.Response.Clear();
        System.Web.HttpContext.Current.Response.BufferOutput = false;  // for large files
        System.Web.HttpContext.Current.Response.ContentType = "application/zip";
        const string filename = "test.zip";
        System.Web.HttpContext.Current.Response.AddHeader("content-disposition", "filename=" + filename);

        ThreadPool.SetMaxThreads(4, 4);
        Log.Info("Starting the batch report");
        using (var zip = new ZipFile())
        {
            var doneEvents = new ManualResetEvent[parameters.Contracts.Length];

            for (int i = 0; i < parameters.Contracts.Length; i++)
            {
                var contract = parameters.Contracts[i];
                doneEvents[i] = new ManualResetEvent(false);
                ThreadPool.QueueUserWorkItem(
                    ExportReport,
                    new ThreadInfo
                    {
                        Contract = contract,
                        Zip = zip,
                        DoneEvent = doneEvents[i]
                    });
            }

            WaitHandle.WaitAll(doneEvents);
            zip.Save(System.Web.HttpContext.Current.Response.OutputStream);
        }

        Log.Info("Finishing the batch report");
    }

protected void ExportReport(object obj)
{
        var info = obj as ThreadInfo;
        var reportDoc = new ReportDocument();
        Log.Info("Thread Id {0} processing contract {1}", Thread.CurrentThread.ManagedThreadId, info.Contract);
        SetPath(reportDoc);
        SetDbInfo(reportDoc);
        LoadParameter(reportDoc);

        reportDoc.SetParameterValue("Contract", info.Contract);
        reportDoc.ExportToDisk(
                ExportFormatType.PortableDocFormat,
                string.Format(@"C:\TempFile\{0}.pdf", info.Contract));
        info.Zip.AddFile(string.Format(@"C:\TempFile\{0}.pdf", info.Contract));
        Log.Info("Thread Id {0} finishing contract {1}", Thread.CurrentThread.ManagedThreadId, info.Contract);
        reportDoc.Dispose();
        info.DoneEvent.Set();          
}

Answer 1

Found it. WaitHandles must be less than or equal to 64.

For solution, please see http://www.codeproject.com/Articles/142341/Solved-The-number-of-WaitHandles-must-be-less-than or Workaround for the WaitHandle.WaitAll 64 handle limit?