CODEWITHSUNDEEP
pythondask
jakirkham
jakirkham

Reputation: 715

Including keyword arguments (kwargs) in custom Dask graphs

Am building a custom graph for one operation with Dask. Am familiar with how to pass arguments to a function in Dask graph and have read up on the docs. However still seem to be missing something.

One of the functions used in the Dask graph takes keyword arguments. Though am confused as to how the keyword arguments can be passed to it. Some of these keyword arguments represent Dask objects so they must be in the graph explicitly (i.e. functools.partial won't work). Can see the following options.

  1. Try to pass any keyword arguments in the correct positional order. (won't work for generic **kwargs or keyword only arguments)
  2. Write a wrapper that converts positional arguments to keywords arguments and use that in the graph. (works ok, but has some overhead)
  3. Include them directly in the graph somehow? (ideal, but unclear how)

Any thoughts on how best to do this?

Upvotes: 4

Views: 688

Answers (2)

MRocklin
MRocklin

Reputation: 57281

A common workaround is to use the apply function

from dask.utils import apply
task = (apply, func, args, kwargs)  # func(*args, **kwargs)

Upvotes: 1

Martin Raspaud
Martin Raspaud

Reputation: 131

The answer by @MRocklin should be edited to conform to newer dask versions:

from dask.utils import apply
task = (apply, func, args, kwargs)  # func(*args, **kwargs)

Upvotes: 0

Related Questions