CODEWITHSUNDEEP
c#multithreadingmontecarlo
John
John

Reputation: 315

Multi-threading in C# (.Net 4.5) for monte-carlo

I have been tasked with taking a monte-carlo model and using multi-threading to make it run faster. As it's monte-carlo, each simulation is independent of the next. I want to manually create each thread as each one will be running many thousands of simulations and storing each simulation result in a database and I want to create as many threads as there are cores in the processor and give them a high priority.

Here is the core code used to manage this (I am testing with a small number of simulations):

        var threads = new List<Thread>();
        iNumCores = Environment.ProcessorCount;
        iSims = 64;

        iNumSimsPerThread = iSims / iNumCores;
        for (int iThread = 0; iThread < iNumCores; iThread++)
        {
            iStart = (iThread * iNumSimsPerThread) + 1;
            iEnd = ((iThread + 1) * iNumSimsPerThread);

            Thread thread = new Thread(() => ProcessParallelMonteCarloTasks(iStart, iEnd, iSims));
            thread.Priority = ThreadPriority.AboveNormal;
            thread.Start();
            threads.Add(thread);
        }
        foreach (var thread in threads)
            thread.Join();

My machine has 8 cores, so this test should create 8 threads, each running 8 simulations. When I write the resultant data to the database, I include the simulation number. I would expect to see 64 rows with 1 per simulation.

However, I get 1 row per simulation up to 40 and then a gap until 57 and then 3 rows for each simulation between 57 and 64.

SQL Server results

Unfortunately, I can't debug a thread and so have no idea what's going on and why it's missing some simulations and running multiple copies of others. When I have task manager open, I can see that 3-4 cores are remaining unused while the program runs.

Any thoughts?

UPDATE: Following Andre's feedback, I looked at the scheduling of threads and the execution of the 'ProcessParallelMonteCarloTasks' function.

The first thing I noticed is that some threads were timing out connecting to the database. I changed the min pool size to equal the number of cores and that fixed that problem, but here is the order in which threads are created and the 'ProcessParallelMonteCarloTasks' gets run:

Execution order

In this instance, the simulation ranges '17-24', '41-48' and '57-64' are called twice and '1-8', '25-32' and '49-56' are missing.

UPDATE 2: I have been watching my Task Manager as this runs and set the thread priority to the highest. What I see suggests that 3 threads are running on 1 core, 2 threads on 2 cores and 1 thread on a fourth core. The other 4 cores are pretty much idle. Is there any way I can get it to run 1 thread per core? The overhead of running 3 threads on a single core must be large.

Upvotes: 1

Views: 1170

Answers (1)

leppie
leppie

Reputation: 117250

iStart and iEnd need to be fresh variables, iow declare them inside your loop, else (they are free) and you end up capturing mutated values as used.

Upvotes: 4

Related Questions