CODEWITHSUNDEEP
pythonpython-multiprocessing
Daniel
Daniel

Reputation: 127

Python pool process management

I was running some Python code and realized when I instantiate Pool with only one process, for instance:

from multiprocessing.pool import Pool
from time import sleep

def f(i):
  print(i)
  sleep(10)

with Pool(1) as p:
    p.map(f, [i for i in range(100)])

Actually five processes are currently running. I've also noticed a pattern going on: if I instantiate Pool with 1, 2,3,... processes the number of processes launched by Python are 5,6,7,... I'm curious: Does Pool use three process for management?

Upvotes: 0

Views: 263

Answers (1)

georgexsh
georgexsh

Reputation: 16624

with Pool(1), you will get 2 processes, the main process (pid 31070), and one worker process (pid 31071), but 3 extra threads/LWPs in the main process (LWP/thread id 31072/31073/31074):

  PID  PPID   LWP  NLWP CMD
31070 21240 31070     4 python3 so_48968836_mp.py
31070 21240 31072     4 python3 so_48968836_mp.py
31070 21240 31073     4 python3 so_48968836_mp.py
31070 21240 31074     4 python3 so_48968836_mp.py
31071 31070 31071     1 python3 so_48968836_mp.py

those three threads are for pool workers maintaining, async task and result handling.

Upvotes: 1

Related Questions