CODEWITHSUNDEEP
pythonanacondadask
Jakub Szlaur
Jakub Szlaur

Reputation: 2132

Creating a Dask client results in endless loop of errors

When I run this code:

from dask.distributed import Client

client = Client(n_workers = 2, threads_per_worker = 2, memory_limit = '2GB', silence_logs='error')
client

I get endless loop of these errors:

tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: None, threads: 2>>
Traceback (most recent call last):
  File "C:\Users\szlau\anaconda3\lib\site-packages\tornado\ioloop.py", line 907, in _run
    return self.callback()
  File "C:\Users\szlau\anaconda3\lib\site-packages\distributed\nanny.py", line 414, in memory_monitor
    process = self.process.process

Note: I have also tried reinstalling the whole Anaconda environment and the error still persists.

I really have no idea what can be causing this or how to eliminate this. Any suggestions are welcomed!

Upvotes: 3

Views: 687

Answers (1)

mdurant
mdurant

Reputation: 28683

At a guess, you are running your code in a script. That means that when dask creates the cluster and associated worker processes, those will, in turn, also runt he script and also attempt to create new clusters and processes.

You should guard your client and cluster creation with

if __name__ == "__main__":
    client = Client(...)

Upvotes: 3

Related Questions