CODEWITHSUNDEEP
pythonparallel-processing
Tom83B
Tom83B

Reputation: 2099

Run many simulations simultaneously

I want to run many random deposition simulations in order to analyze the statistics.

The individual simulations aren't dependent on each other, but each produces a 1-D numpy array and I want my result as a 2-D array composed of these 1-D arrays.

I know basically nothing about parallel computing so I have only have two ideas how to achieve this:

  1. Simple for-loop which will take days to run (I'm not in hurry after all)
  2. Run the same file many, each with shorter for-loop, let OS take care of distributing CPU power among the runs and join the files after everything is finished

However, I think that there must be a better way to do this. My current code is following:

L = 60
N = 100
T = 100

hrubost_relax = np.zeros((N,T))

for n in range(0,N):
    level = np.zeros((T,L))
    heights = np.zeros(level.shape[1])

    for t in range(1,level.shape[0]):
        pos = np.random.randint(level.shape[1])

        left = heights[pos-1] if pos-1>0 else heights[-1]
        right = heights[pos+1] if pos+1<L else heights[0]

        if left<heights[pos]:
            if right<heights[pos]:
                direction = np.random.randint(2)*2-1
                heights[(pos+direction)%L] += 1
            else:
                heights[pos-1] += 1
        elif right<heights[pos]:
            heights[(pos+1)%L] += 1
        else:
            heights[pos] += 1

        hrubost_relax[n,t] = heights.std()/L

I want to parallelize the outer for-loop

edit to show my solution with multiprocessing.Pool

from multiprocessing import Pool

L = 60
N = 1000
T = 100000

hrubost_relax = np.zeros((N,T))

def deposition(n):
    level = np.zeros((T,L))
    heights = np.zeros(level.shape[1])
    w = np.zeros(T)

    for t in range(1,level.shape[0]):
        pos = np.random.randint(level.shape[1])

        left = heights[pos-1] if pos-1>0 else heights[-1]
        right = heights[pos+1] if pos+1<L else heights[0]

        if left<heights[pos]:
            if right<heights[pos]:
                direction = np.random.randint(2)*2-1
                heights[(pos+direction)%L] += 1
            else:
                heights[pos-1] += 1
        elif right<heights[pos]:
            heights[(pos+1)%L] += 1
        else:
            heights[pos] += 1

        w[t] = heights.std()/L

    return w

p = multiprocessing.Pool(4)
for i,x in enumerate(p.map(deposition, range(N))):
    hrubost_relax[i] = x

Upvotes: 1

Views: 146

Answers (1)

holdenweb
holdenweb

Reputation: 37023

Threading cannot give you any speedup on this problem if you are using the CPython implementation due to its use of a global interpreter lock, which only allows a single thread to be executing at any time (I am assuming the computations are CPU-bound). If you have multiple processors in your computer, however, the multiprocessing module will let you run a number of processes in parallel. There would still be no real value to trying to run more processes than you have processors for, however, so you should consider using multiprocessing.Pool

Upvotes: 2

Related Questions