CODEWITHSUNDEEP
parallel-processingneural-networkconv-neural-networkgoogle-colaboratorydropout
Cosapocha
Cosapocha

Reputation: 11

Parallel computing in Colab

I'm trying to do some parallel computing (frist time ever!) and I don't really know how to do it, or if it is going to speed up my computation.

I have a neural net in a Colab Notebook and I have to run through it the same minibatch of images N times, in order to do some dropout statistics.

It is probably quite a simple task, but I have no idea how to do it. The code would be as easy as:

for i in np.arange(iters):
   sample[i] = model(x)

Or something like that, you get the idea. The thing is that model (x) consumes quite a lot of time, and I would really need to do it in parallel.

Also, a somewhat related question: how many cores does Colab have? The iters should be in the order of 10.000 so, it is probably way too much, isn't it?

Upvotes: 1

Views: 2429

Answers (1)

Fábio Castro
Fábio Castro

Reputation: 11

This is a simple example on how to paralelize in Colab: https://colab.research.google.com/drive/1uDd7eq6dAlHGW9o7B-IDResAzShVjh1Z?usp=sharing

from multiprocessing import Pool
import time

def wait_fn(t):
  time.sleep(t/10)

data = [1]*10

# Only one thread
start = time.time()
out = [wait_fn(t) for t in data]
end = time.time()
print("Only one thread: ",end - start,"s")

# Palalelize
start = time.time()
with Pool() as p:
  out = p.map(wait_fn, data)
  end = time.time()
print("Paralelized: ",end - start,"s")

Only one thread: 1.0021319389343262 s

Paralelized: 0.6200098991394043 s

However, this seems to use only two cores. I'm still trying to find out how to use more cores, if possible.

Upvotes: 1

Related Questions