CODEWITHSUNDEEP
concurrencyparallel-processing.net-coref#
MHilton
MHilton

Reputation: 221

Parallel async tasks not executing concurrently

I am trying to parallelise a set simulations of multi agent systems so they can utilise as many cpu cores as are available to me (currently 72). To do this I am trying to package each simulation as a separate asynchronous computation and then running them in parallel.

The following code is how I run the simulation. SimulationLst is a list of simulation initial states. Each simulation returns a list of integers which I then average over all the simulations. Each simulation has no side effects.

SimulationList
|> List.map (fun simulation -> async {return runSimulation simulation})
|> Async.Parallel
|> Async.RunSynchonously
|> Aray.toList
|> List.concat
|> List.average

The problem is that when I run the program, the first four simulations start immediately but the next start very slowly one after another. The result is that the cpu utilisation starts off very poor and very slowly ramps up to use more cores.

What reasons could there be for these computations not starting immediately? Is it because I am doing this at quite a high level (ie simulation by simulation)? Would finer grain concurrency make this work better?

There is not much detail in this question about the code I'm using since there is a lot of it but please ask for more detail if it would help.

Upvotes: 2

Views: 76

Answers (1)

Chester Husk
Chester Husk

Reputation: 1473

My guess would be that you're in a situation where the ThreadPool has a limited number of threads available to process tasks and so is slowly ramping up the thread count at a rate of 0.5/sec or 1/sec. You should try adjusting the minimum ThreadPool thread count before running your code to see if that alleviates the problem.

Upvotes: 4

Related Questions