CODEWITHSUNDEEP
daskdask-distributed
ericmjl
ericmjl

Reputation: 14724

Dask: set multiprocessing method from Python

Is there a way to set the multiprocessing method from Python? I do not see a method in the Client() API docs of Dask.distributed that indicates how to set this property.

Update:

For example, is there:

client = Client(multiprocessing='fork')

or

client = Client(multiprocessing='spawn')

?

Upvotes: 4

Views: 1305

Answers (1)

MRocklin
MRocklin

Reputation: 57319

Unfortunately the multiprocessing context method is set at import time of dask.distributed. If you wanted to set this from Python you could set the config value after you import dask, but before you import dask.distributed.

import dask
dask.config.set({'distributed.worker.multiprocessing-method': 'spawn'})

from dask.distributed import Client

However it's probably more robust to just set this in your config file. See configuration documentation for the various ways to set configuration values.

Note: this is using the configuration as of dask.__version__ == '0.18.0'

Upvotes: 4

Related Questions