CODEWITHSUNDEEP
pythondask
Bertrand
Bertrand

Reputation: 1064

Dask worker persistent variables

Is there a way with dask to have a variable that can be retrieved from one task to another. I mean a variable that I could lock in the worker and then retrieve in the same worker when i execute another task.

Upvotes: 2

Views: 510

Answers (1)

MRocklin
MRocklin

Reputation: 57311

The workers themselves are just Python processes, so you could do tricks with globals().

However, it is probably cleaner to emit values and pass these between tasks. Dask retains the right to rerun functions and run them on different machines, so depending on global state or worker-specific state can easily get you into trouble.

Upvotes: 1

Related Questions