CODEWITHSUNDEEP
pythonlistsubprocess
Daedalus Mythos
Daedalus Mythos

Reputation: 575

Keep a variable number of subprocesses constantly running

I'm planning a list of subprocesses of some.exe e.g.

max_processes = 4
proc_list = []
for i in range(100):
    while len(proc_list) <= max_processes:
        proc_list.append(subprocess.Popen(["some.exe", i]))

The processes should work in the background and the number of max_processes should constantly run. The std out of some.exe is not required.

This is a rough plan of what I'd like to do, is this even possible or do I have to rethink the whole concept?

Upvotes: 0

Views: 62

Answers (1)

falsetru
falsetru

Reputation: 369194

You can use multiprocessing.Pool or multiprocessing.pool.ThreadPool.

from multiprocessing import Pool
# OR  from multiprocessing.pool import ThreadPool as Pool
import subprocess

def run_some_exe(i):
    subprocess.call(['some.exe', i])

if __name__ == '__main__':
    max_processes = 4
    pool = Pool(max_processes)
    pool.map(run_some_exe, range(100))

mulitprocess.Pool.map works like map except that it run the function parallelly. (max 4 in above example)

Upvotes: 2

Related Questions