CODEWITHSUNDEEP
dask
Arco Bast
Arco Bast

Reputation: 3892

How to specify the directory that dask uses for temporary files?

Dask seems to write to the /tmp folder. How can I change the folder that dask uses for temporary files?

Upvotes: 10

Views: 5772

Answers (3)

danodonovan
danodonovan

Reputation: 20373

Setting TMPDIR could potentially cause problems as it might also effect other applications. An alternative is to use dask.config.set

>>> import dask
>>> with dask.config.set({'temporary_directory': '/path/to/tmp'}):
...     pass

You could also add the lines

temporary_directory: /path/to/tmp

to ~/.config/dask/distributed.yaml (in your home directory - see configuration docs)

Upvotes: 8

Alexei Andreev
Alexei Andreev

Reputation: 675

For some reason the accepted code doesn't work for me. I get this error: AttributeError: 'set' object has no attribute 'items'

Here's a version that works: dask.config.set(temporary_directory='/path/to/tmp')

Also note that you have to do this before you create your Client.

Upvotes: 6

Arco Bast
Arco Bast

Reputation: 3892

Setting the TMPDIR environment variable to the desired location via export TMPDIR=/my/path seems to work.

Upvotes: 1

Related Questions