python multiple processing using pool

Question

I have a assignment for my course in Python.

The question is this :

You will write a program creating multiple processes (p=10). Each process will receive integer value (y) and compute y multiplying by 4 (y*4). A collection of y values is a list of integers [0, 1, 2, 3, 4, 5]. You must use a process pool, so you need to map these 10 processes to take the list of y integers. Do the following:

Your program should print out the following three. (1) Print out the original list. (2) Print each running process name and its output of y*4 in random order of execution

Output of (1): Input list: [0, 1, 2, 3, 4, 5]

Output of (2): Output in random order:

PoolWorker-10 output: [0, 4, 8, 12, 16, 20]

PoolWorker-11 output: [0, 4, 8, 12, 16, 20]

PoolWorker-12 output: [0, 4, 8, 12, 16, 20]

My first attempt was:

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(number):
    result=4*number
    Process_name=current_process().name
    print(f"Poolworker -{Process_name} output: {result} ")

    return result

if __name__=='__main__':
    p= Pool(processes=10)
    numbers=[0,1,2,3,4,5]
    result=p.map(f,numbers)

The output was:

Poolworker -SpawnPoolWorker-2 output: 0 
Poolworker -SpawnPoolWorker-2 output: 4 
Poolworker -SpawnPoolWorker-2 output: 8 
Poolworker -SpawnPoolWorker-2 output: 12 
Poolworker -SpawnPoolWorker-2 output: 16 
Poolworker -SpawnPoolWorker-2 output: 20 

My second attempt:

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(*number):
	numbers=list(number)
	i=0
	for x in numbers:
		numbers[i]=4*numbers[i]
		i+=1
	Process_name=current_process().name
	print(f"Poolworker -{Process_name}output: {numbers} ")

	return numbers

if __name__ == '__main__':
		array=[0,1,2,3,4,5]
		p=Pool(processes=10)
	
		result=p.map(f,array)
		
               

I still have the wrong output with the changes.

How can I get the desired output?

Answer 1

The issue is that f is applied to each number in your list whereas you want every worker to do the calculations on the whole list. Made some minor modifications to your code in order to get those results.

from itertools import repeat
from multiprocessing import Pool
from multiprocessing import Process, current_process

def f(numbers):
    result= [4 * num for num in numbers]
    name=current_process().name
    print(f"Poolworker - {name} output: {result} ")
    return result

if __name__=='__main__':
    WORKERS = 10
    p = Pool(processes=WORKERS)
    numbers = [0, 1, 2, 3, 4, 5]
    print(f"Original - {numbers}")
    result = p.map(f, repeat(numbers, WORKERS))

This returns the following:

Original - [0, 1, 2, 3, 4, 5]
Poolworker - ForkPoolWorker-1 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-2 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-3 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-4 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-5 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-6 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-7 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-8 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-9 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-10 output: [0, 4, 8, 12, 16, 20]

Answer 2

From your description you want all processes to multiply array by 4.

So use parallel processes that are each given an array of numbers to multiply.

Reference

Code

from multiprocessing import Process

import multiprocessing as mp

def f(numbers, output):
    " Places process name and numbers multiplied by 4 into output queue "
    output.put((mp.current_process().name, [4*n for n in numbers]))

# Define an output queue
output = mp.Queue()

# Setup a list of processes that we want to run
numbers = [1, 2, 3, 4, 5]
processes = [mp.Process(target=f, args=([1, 2, 3, 4, 5], output)) for x in range(10)]

# Run processes
for p in processes:
    p.start()

# Exit the completed processes
for p in processes:
    p.join()

# Get process results from the output queue
results = [output.get() for p in processes]


for id, values in results:
  print(f'PoolWorker {id} output: {values}')

Output

PoolWorker Process-1 output: [4, 8, 12, 16, 20]
PoolWorker Process-3 output: [4, 8, 12, 16, 20]
PoolWorker Process-9 output: [4, 8, 12, 16, 20]
PoolWorker Process-6 output: [4, 8, 12, 16, 20]
PoolWorker Process-7 output: [4, 8, 12, 16, 20]
PoolWorker Process-5 output: [4, 8, 12, 16, 20]
PoolWorker Process-4 output: [4, 8, 12, 16, 20]
PoolWorker Process-8 output: [4, 8, 12, 16, 20]
PoolWorker Process-2 output: [4, 8, 12, 16, 20]
PoolWorker Process-10 output: [4, 8, 12, 16, 20]