CODEWITHSUNDEEP
c#.netmultithreadingmemory-leakstask
erdem
erdem

Reputation: 121

Multithreaded Application and Potential Memory Leak (C#)

I have a console application that runs jobs in a scheduled fashion. A job does 2 things:

1-) Run a SQL statement 2-) E-mail the result of that statement

If I run the job in sequential form, everything works as expected, a job runs, memory consumption increases during the operation and then memory is released, however if I run the jobs in parallel, using the task parallel library, after all the jobs finish, memory consumption stays much higher compared to the sequential option and additional jobs also continues to increase memory consumption.

To be more specific I used the following test cases:

Sequential: (After loop is completed and GC is collected explicitly for test purposes memory consumption is around 55 megabytes)

for (int j = 0; j < 3; j++)
{
    for (int i = 0; i < 15; i++)
    {
        var job = new BIJob(reportData);
        job.Execute();
    }
    Thread.Sleep(10000);
}

Parallel:(After loop is completed and GC is collected explicitly for test purposes memory consumption is around 85 megabytes)

for (int j = 0; j < 3; j++)
{
    for (int i = 0; i < 15; i++)
    {
        Task jobRunTask = Task.Factory.StartNew(() =>
        {
            var job = new BIJob(reportData);
            job.Execute();

        });
    }
    Thread.Sleep(10000);
}

There is approximately 30 megabytes difference in memory consumption after 45 iterations and that extra memory is not being collected in the parallel version.

What might be causing this kind of behaviour? Any ideas/comments is appreciated.

Upvotes: 1

Views: 1856

Answers (2)

Foxfire
Foxfire

Reputation: 5755

The Task parallel library will simply keep some of the threads it creates around in case it will need them later because creating new Threads is a relatively costly operation (both in terms of memory and CPU).

Regarding the memory-leak: As long as there is no stress on resources there is no reason for the threadpool used by the TPL to free any threads. If you want to test for a memory-leak you can simply increase the count of your loop. There should be no difference in memory-usage after say looping 1000 times or 1000000 times.

Upvotes: 0

Servy
Servy

Reputation: 203802

When you're performing multiple operations in parallel you'll need to store enough memory to work on each of those parallel operations, rather than having the working set of just one in memory at a time. You also have additional threads, each of which are going to consume memory.

The memory for those operations won't be able to be reclaimed until they actually finish. You're just starting the operations in your loop, not waiting for them to finish, so they are not necessarily even eligible for collection whenever it is that you're checking on them. If you wait for all of the operations to actually finish then they will be eligible for collection, although the GC is of course free to wait as long as it wants to actually collect them.

Upvotes: 1

Related Questions