CODEWITHSUNDEEP
pythonclassmultiprocessingreturn
fleshstorm
fleshstorm

Reputation: 67

Python - how to use multiprocessing with class instance

currently I run a loop in a loop in a loop that then pass a new set of parameters of an already instanciated class and then call at first a class.reset() function and then the class.main() function.

As class.run is very cpu intense I do want to multiprocess this, but nowhere I've found an example how to do this.

Below is the code that needs to be multiprocessed:


st = Strategy()   /// Strategy is my class

for start_delay in range(0, PAR_BT_CYCLE_LENGTH_END, 1):
    for cycle_length in range(PAR_BT_CYCLE_LENGTH_START, PAR_BT_CYCLE_LENGTH_END+1, 1):
        for cycle_pos in range(PAR_BT_N_POS_START, PAR_BT_N_POS_END+1, 1):

            st.set_params(PAR_BT_START_CAPITAL, start_delay, cycle_length, cycle_pos, sBT,
                          iPAR_BT_TF1, iPAR_BT_TF2, iPAR_BT_TF3, iPAR_BT_TF4,
                          iPAR_BT_TFW1, iPAR_BT_TFW2, iPAR_BT_TFW3, iPAR_BT_TFW4)
            st.reset()                
            bt = st.main()

# do something with return values (list) in bt

# after all processes have finished - use return values of all processes

What would be the best way to get this working as multiple processes?

Upvotes: 0

Views: 229

Answers (1)

James
James

Reputation: 36608

You can use the ProcessPoolExecutor from concurrent.futures.

from concurrent.futures import ProcessPoolExecutor, as_completed

def run_strategy(*args):
    st = Strategy(
    st.set_params(*args)
    st.reset()
    bt = st.main()
    return bt

ex = ProcessPoolExecutor()
futures = []
for start_delay in range(0, PAR_BT_CYCLE_LENGTH_END, 1):
    for cycle_length in range(PAR_BT_CYCLE_LENGTH_START, PAR_BT_CYCLE_LENGTH_END+1, 1):
        for cycle_pos in range(PAR_BT_N_POS_START, PAR_BT_N_POS_END+1, 1):
            args = (
                PAR_BT_START_CAPITAL, 
                start_delay, 
                cycle_length, 
                cycle_pos, 
                sBT,
                iPAR_BT_TF1, 
                iPAR_BT_TF2, 
                iPAR_BT_TF3, 
                iPAR_BT_TF4,
                iPAR_BT_TFW1, 
                iPAR_BT_TFW2, 
                iPAR_BT_TFW3, 
                iPAR_BT_TFW4
            )
            ex.submit(run_strategy, *args)

# collect the returned bts
bt_results = []
for f in as_completed(futures):
    bt_results.append(f.result())

ex.shutdown()

Upvotes: 1

Related Questions