CODEWITHSUNDEEP
pythonmultiprocessing
jrsm
jrsm

Reputation: 1695

Suppress output in multiprocessing process

I have some code which runs in parallel using the multiprocessing Pool class. Unfortunately some of the functions I use from another library have some verbose output. To abstract the problem have a look at the following example:

from multiprocessing import Pool

def f(x):
    print 'hello'
    return x*x

p = Pool(10)
p.map(f, [1,2,3])

So this will print 'hello' like 10 times. Is there a possibility to mute the output of the processes or to put the std.out in some variable? Thanks for any suggestion.

EDIT: I don't want to redirect the whole stdout, just for the processes of my pool.

Upvotes: 5

Views: 6174

Answers (1)

unutbu
unutbu

Reputation: 880887

Use the initializer parameter to call mute in the worker processes:

import sys
import os
from multiprocessing import Pool

def mute():
    sys.stdout = open(os.devnull, 'w')    

def f(x):
    print 'hello'
    return x*x

if __name__ == '__main__':
    p = Pool(10, initializer=mute)
    p.map(f, [1,2,3])
    print('Hello')

prints

Hello

Upvotes: 11

Related Questions