CODEWITHSUNDEEP
daskdask-distributed
Dustin van Weersel
Dustin van Weersel

Reputation: 35

Seeing logs of dask workers

I'm having trouble changing the temporary directory in Dask. When I change the temporary-directory in dask.yaml for some reason Dask is still writing out in /tmp (which is full). I now want to try and debug this, but when I use client.get_worker_logs() I only get INFO output.

I start my cluster with

from dask.distributed import LocalCluster, Client

cluster = LocalCluster(n_workers=1, threads_per_worker=4, memory_limit='10gb')

client = Client(cluster)

I already tried adding distributed.worker: debug to the distributed.yaml, but this doesn't change the output. I also check I am actually changing the configuration by calling dask.config.get('distributed.logging')

What am I doing wrong?

Upvotes: 2

Views: 1490

Answers (1)

MRocklin
MRocklin

Reputation: 57251

By default LocalCluster silences most logging. Try passing the silence_logs=False keyword

cluster = LocalCluster(..., silence_logs=False)

Upvotes: 4

Related Questions