CODEWITHSUNDEEP
c#
smned
smned

Reputation: 683

Run method with parameters in parallel

I have a method which takes an argument and run it against database, retrieve records, process and save processed records to a new table. Running the method from the service with one parameter works. What i am trying to achieve now is make the parameter dynamic. I have implemented a method to retrieve the parameters and it works fine. Now i am trying to run methods parallel from the list of parameter's provided. My current implementation is:

WorkerClass WorkerClass = new WorkerClass();
var ParametersList = WorkerClass.GetParams();

foreach (var item in ParametersList){
    WorkerClass WorkerClass2 = new WorkerClass();
    Parallel.Invoke(
        ()=>WorkerClass2.ProcessAndSaveMethod(item)
    );
}

On the above implementation i think defining a new WorkerClass2 defies the whole point of Parallel.Invoke but i am having an issue with data mixup when using already defined WorkerClass. The reason for the mix up is Oracle connection is opened inside the Init() Method of the class and static DataTable DataCollectionList; is defined on a class level thus creating an issue.
Inside the method ProcessAndSaveMethod(item) i have: 

OracleCommand Command = new OracleCommand(Query, OracleConnection);             
            OracleDataAdapter Adapter = new OracleDataAdapter(Command);
            Adapter.Fill(DataCollectionList);

Inside init(): 

try
{
    OracleConnection = new OracleConnection(Passengers.OracleConString);
    DataCollectionList = new DataTable();
    OracleConnection.Open();
    return true;
}
catch (Exception ex)
{
    OracleConnection.Close();
    DataCollectionList.Clear();
    return false;
 }

And the function isn't run parallely as i was trying to do. Is there another way to implement this?

Upvotes: 0

Views: 1314

Answers (3)

smned
smned

Reputation: 683

I achieved parallel processing using the below code and also avoided null pointer exception from DbCon.open() caused by connection pooling using the max degree of parallelism parameter.

Parallel.ForEach(ParametersList , new ParallelOptions() { MaxDegreeOfParallelism = 5 }, item=>
            {
                WorkerClass Worker= new WorkerClass();
                Worker.ProcessAndSaveMethod(item);
            });

Upvotes: 0

nvoigt
nvoigt

Reputation: 77304

If you have a list of somethings and want it processed in parallel, there really is no easier way than PLinq:

var parametersList = SomeObject.SomeFunction();

var resultList = parametersList.AsParallel()
                               .Select(item => new WorkerClass().ProcessAndSaveMethod(item))
                               .ToList();

The fact that you build up a new connection and use a lot of variables local to the one item you process is fine. It's actually the preferred way to do multi-threading: keep as much local to the thread as you can.

That said, you have to measure if multi-threading is actually the fastest way to solve your problem. Maybe you can do your processing sequentially and then do all your database stuff in one go with bulk inserts, temporary tables or whatever is suited to your specific problem. Splitting a task into smaller tasks for more processors to run is not always faster. It's a tool and you need to find out if that tool is helping in your specific situation.

Upvotes: 1

mr100
mr100

Reputation: 4428

To run it in parallel you need to call Parallel.Invoke only once with all the tasks to be completed:

Parallel.Invoke(
    ParametersList.Select(item =>
        new Action(()=>WorkerClass2.ProcessAndSaveMethod(item))
    ).ToArray()
);

Upvotes: 1

Related Questions