CODEWITHSUNDEEP
daskdask-distributed
Nuno Silva
Nuno Silva

Reputation: 116

Dask Client detect local default cluster already running

from dask.distributed import Client

Client()
Client(do_not_spawn_new_if_default_address_in_use=True)  # should not spawn a new default cluster

Is this possible to do somehow?

Upvotes: 1

Views: 3120

Answers (2)

mdurant
mdurant

Reputation: 28673

The non-public function distributed.client._get_global_client() will return the current client if it exists, or None

client = _get_global_client() or Client()

Since it is internal, the API may change without notice.

Upvotes: 7

Jacob Tomlinson
Jacob Tomlinson

Reputation: 3773

You shouldn't really be creating multiple clients within the same Python session. It may be worth digging deeper into why you are calling Client() more than once.

If you already have a Dask cluster running on the default address you could set the DASK_SCHEDULER_ADDRESS environment variable which will instruct the client to look there instead of creating a local cluster.

>>> import os
>>> os.environ['DASK_SCHEDULER_ADDRESS'] = 'tcp://localhost:8786'

>>> from dask.distributed import Client
>>> Client() # Does not create a cluster
    <Client: 'tcp://127.0.0.1:8786' processes=0 threads=0, memory=0 B>

Upvotes: 2

Related Questions