CODEWITHSUNDEEP
pythonconcurrencytwistedreactor
NNamed
NNamed

Reputation: 727

Python - simplest way possible to run functions in parallel

I'm struggling a bit with something that should actually be quite simple to do. I have a function that does some long computation, to simplify it loks like this: 

import time    
def compute_stuff(number):
   time.sleep(10)
   return [number*2, number*4]

I want to run TWO instances of this function in parallel and collect their results in one array. I've read a bit about Twisted reactor and it seems to provide async queries, but is running everything in one thread so when I do e.g.:

from twisted.internet import reactor
import time

def compute_stuff(number):
   time.sleep(10)
   return [number*2, number*4]

reactor.callWhenRunning(compute_stuff, 1)
reactor.callWhenRunning(compute_stuff, 4)
reactor.callWhenRunning(compute_stuff, 2)
reactor.run()

It waits for the first call to complete before executing the next one. Is there a way to make it parallel? Is Twisted even the way to go?

Upvotes: 0

Views: 1043

Answers (2)

LearnerEarner
LearnerEarner

Reputation: 1020

You can try the threading module

import threading
import time    

def compute_stuff(number):
   print str(number) + "start"
   time.sleep(10)
   print str(number) + "end"
   return [number*2, number*4]


threads = []
for i in range(5):
    t = threading.Thread(target=compute_stuff, args=(i,))
    threads.append(t)
    t.start()

Upvotes: 2

Filip
Filip

Reputation: 128

With task You can run them in intervals like:

from twisted.internet import task
from twisted.internet import reactor
import time

def compute_stuff(number):
   time.sleep(10)
   return [number*2, number*4]

t1 = task.callWhenRunning(compute_stuff, 1)
t2 = task.callWhenRunning(compute_stuff, 4)
t3 = task.callWhenRunning(compute_stuff, 2)

t1.start(10) #run every 10 seconds
t2.start(10)
t3.start(10)

reactor.run()

But I'm not sure about threads...

Upvotes: 0

Related Questions