CODEWITHSUNDEEP
pythondistributeddask
Alexander Kislitsky
Alexander Kislitsky

Reputation: 13

Is it possible to run dask/distributed jobs on the certain workers list

I have a computation network with 40 workers, but I need to perform calculations only on certain hosts. Is it possible to limit or rebuild network for using only specified workers?

Upvotes: 1

Views: 713

Answers (1)

MRocklin
MRocklin

Reputation: 57271

Yes, you can specify particular workers with the workers= keyword to client.submit, client.map, client.compute, or client.persist.

futures = client.map(func, args, workers=['worker-1', 'worker-2'])

See http://distributed.readthedocs.io/en/latest/locality.html#user-control for more options

You can also register resources like GPUs with the workers and restrict computations to those. See http://distributed.readthedocs.io/en/latest/resources.html for more information.

Upvotes: 1

Related Questions