CODEWITHSUNDEEP
pythonpython-3.xlistloopsparallel-processing
Nkolot
Nkolot

Reputation: 87

Create different processes using a list of objects

I want to execute this function without having to rewrite all the code for each process.

def executeNode(node):
    node.execution()

And the code that I don't feel the need to repeat n times the next one. I need to use Process not Threads.

a0 = Process(target=executeNode, args = (node1))
a1 = Process(target=executeNode, args = (node2))
a2 = Process(target=executeNode, args = (node3))
...............................
an = Process(target=executeNode, args = (nodeN))

So I decided to create a list of nodes but I don't know how to execute a process for each item (node) of the list. 

sNodes = []

for i in range(0, n):
    node = node("a"+ str(i), (4001 + i))
    sNodes.append(node)

How can I execute a process for each item (node) of the list (sNodes). Thank you all.

Upvotes: 0

Views: 60

Answers (2)

Thomas Milox
Thomas Milox

Reputation: 104

Try something like this:

from multiprocessing import Pool

process_number = 4
nodes = [...]

def execute_node(node):
    print(node)

pool = Pool(processes=process_number)
pool.starmap(execute_node, [(node,) for node in nodes])
pool.close()

You will find more intel here: https://docs.python.org/3/library/multiprocessing.html

Upvotes: 0

Netwave
Netwave

Reputation: 42756

You can use a Pool:

from multiprocessing import Pool

if __name__ == '__main__':
    with Pool(n) as p:
        print(p.map(executeNode, sNodes))

Where n is the number of processes you want.

In case you want detached processes or you dont expect a result is better to simply use another loop:

processes = []
for node in sNodes:
    p = Process(target=executeNode, args = (node1))
    processes.append(p)
    p.Start()

General tip: having a lot of processes will not speed up your code but make your processor start swaping and everything will be slower. Just in case you are looking for a code speedup instead of a logical architecture.

Upvotes: 3

Related Questions